AI Engineering Living Glossary #012 — Human-in-the-Loop: Designing Human Oversight into AI Systems

AI ENGINEERING | LIVING GLOSSARY | #012

HUMAN-IN-THE-LOOP (HITL)

Designing Human Oversight into AI Systems

Chapter 2: Engineering AI for Reliability

AIYING — Accomplishing with AI

An AI assistant processes employee reimbursement claims. It reads receipts, checks company policy, and recommends whether each claim should be approved.

For routine claims, the assistant appears accurate and efficient.

Then it encounters a ₹75,000 claim with incomplete supporting documents and an unusual expense category.

Should the AI approve it? Reject it? Or ask a human to decide?

This is not simply a question about model accuracy. It is a question about how responsibility and authority are designed into an AI system.

Engineering insight: Human-in-the-Loop is not merely placing a human after an AI model. It is designing meaningful opportunities for human judgment, intervention, and authorization at appropriate points in the workflow.

Contents

1. What Is Human-in-the-Loop?
2. Why HITL Matters
3. Human Oversight Patterns
4. How HITL Works
5. When Should Humans Intervene?
6. Engineering HITL Architecture
7. Industry Applications
8. Engineering Decisions
9. Practical Industry Case
10. Try It Yourself
11. Connected Glossary Terms
12. Living Update
13. Key Takeaway
14. References

1. What Is Human-in-the-Loop?

Human-in-the-Loop (HITL) describes a system design in which human participation is deliberately incorporated into an automated process.

In AI applications, people may review outputs, provide corrections, resolve uncertainty, approve actions, or make final decisions.

HITL may involve:

• Reviewing AI-generated recommendations.
• Correcting inaccurate information.
• Providing additional context.
• Resolving ambiguous cases.
• Approving consequential actions.
• Rejecting or overriding proposed decisions.
• Escalating exceptional cases.

Human participation can occur during development, training, evaluation, or production. This lesson focuses primarily on operational oversight in deployed AI applications.

Human oversight is a broader governance concept. HITL is one possible technical and organizational approach to implementing it.

↑ Back to contents

2. Why Human-in-the-Loop Matters

AI applications can process information quickly, but speed and scale do not eliminate uncertainty, errors, or accountability requirements.

Human judgment can be especially important when decisions have significant consequences.

Engineering challenge	Potential HITL contribution
Incomplete information	Request clarification or additional evidence.
Ambiguous policy interpretation	Route to an authorized reviewer.
High-impact action	Require approval before execution.
Unexpected model behavior	Enable investigation and intervention.
Conflicting evidence	Allow informed resolution.
Model errors	Support correction and feedback.

However, human involvement does not automatically make a system reliable.

Reviewers may lack information, have insufficient time, or over-trust AI recommendations. Meaningful oversight requires suitable interfaces, authority, competence, and working conditions.

↑ Back to contents

3. Human Oversight Patterns

A. Human-in-the-Loop

A human participates at defined stages of an AI workflow, such as reviewing a proposed transaction before execution.

B. Human-on-the-Loop

A human supervises an operating system and can intervene when necessary. The degree of practical control depends on system design and response time.

C. Human-out-of-the-Loop

The system operates without routine human intervention during the relevant process. This may be suitable for some low-risk tasks but requires appropriate risk assessment.

D. Human approval

Designated actions remain blocked until an authorized person explicitly approves them.

E. Human escalation

Uncertain, exceptional, or high-risk cases are routed to a person with the appropriate responsibility.

F. Human correction

Reviewers can edit, reject, or replace AI-generated outputs.

Important distinction: Human approval, human monitoring, and human intervention are different control mechanisms. They should not be treated as interchangeable.

↑ Back to contents

4. How HITL Works

Consider an AI-assisted expense approval process.

Employee Submits Claim
          |
          v
AI Extracts and Checks Information
          |
          v
Policy and Risk Checks
          |
          v
Decision Routing
      /          \
     v            v
Routine Case   Review Required
     |            |
     v            v
Permitted      Human Reviews
Processing        |
     |       Approve / Reject /
     |       Request Information
     |            |
     v            v
Authorized Workflow Continues
          |
          v
Audit Record and Feedback

In this illustrative design, routine processing does not mean unrestricted autonomous payment approval. Any action requiring authorization remains subject to the applicable controls.

The key design question is where human intervention provides meaningful value and where it is mandatory.

↑ Back to contents

5. When Should Humans Intervene?

Human review can be triggered by risk, uncertainty, policy, or the nature of a proposed action.

A. High-impact decisions

Examples include financial approvals, employment decisions, and other consequential outcomes.

B. Missing or conflicting evidence

Escalate cases where the available information cannot adequately support a decision.

C. Policy exceptions

Require authorized review when a case falls outside established rules.

D. Irreversible actions

Consider explicit approval before operations that are difficult or impossible to undo.

E. Unusual patterns

Route anomalous or potentially fraudulent cases for further investigation.

F. Regulatory or organizational requirements

Implement human oversight where required by applicable law, policy, or governance standards.

A model's confidence score alone is not a sufficient basis for deciding when humans should intervene. Scores may be poorly calibrated and must be validated for the application.

↑ Back to contents

6. Engineering HITL Architecture

Effective HITL systems require more than a review screen.

1. Decision classification

Identify which requests require human review and which can follow authorized automated workflows.

2. Review queue

Route cases to the appropriate reviewers based on responsibility, priority, and expertise.

3. Evidence presentation

Show relevant inputs, sources, policy rules, uncertainty, and the proposed AI action.

4. Reviewer controls

Provide meaningful options to approve, reject, correct, request information, or escalate.

5. Authorization enforcement

Verify reviewer identity and permissions before allowing consequential actions.

6. Execution control

Prevent the proposed operation from executing before the required approval is obtained.

7. Audit trail

Record decisions, responsible roles, timestamps, and relevant supporting evidence while respecting privacy requirements.

8. Feedback and evaluation

Use review outcomes to identify recurring errors and improve system design, with appropriate data governance.

Engineering principle: A human reviewer must have the information, authority, time, and practical ability to challenge the AI's recommendation.

↑ Back to contents

7. Industry Applications

AI application	Human role
Financial operations	Approve transactions and investigate exceptions.
Healthcare support	Qualified professionals assess clinical recommendations.
Recruitment support	Authorized personnel review consequential assessments.
AI coding agents	Review changes and authorize sensitive deployments.
Customer service	Handle escalations and policy exceptions.
Legal document analysis	Qualified reviewers validate interpretations.
Enterprise AI agents	Authorize designated external actions.

The appropriate oversight pattern depends on the task, consequences, applicable requirements, and available human expertise.

↑ Back to contents

8. What Must an AI Engineer Decide?

A. Which decisions require human involvement?

Classify workflows according to impact, reversibility, uncertainty, and policy requirements.

B. Who is authorized to review?

Define reviewer roles, qualifications, and approval permissions.

C. What information does the reviewer need?

Present evidence, context, relevant rules, and the proposed action clearly.

D. Can the reviewer genuinely intervene?

Ensure the reviewer can reject, modify, or stop the operation.

E. What happens when nobody responds?

Define timeouts, escalation, and safe fallback behavior.

F. How will review workload be managed?

Consider queue capacity, reviewer fatigue, turnaround time, and staffing.

G. How will disagreement be handled?

Define procedures for conflicting assessments and exceptional cases.

H. How will oversight quality be measured?

Track review accuracy, override patterns, processing time, and missed escalations.

I. How will the system learn?

Use reviewed outcomes to improve evaluations and workflows, without assuming that every human decision is correct.

↑ Back to contents

9. Practical Industry Case: The Claim That Needed a Human

An enterprise AI assistant supports travel reimbursement processing.

It receives a claim for ₹75,000 containing several expense items, incomplete receipts, and a request for an exception to company policy.

The AI can extract information and prepare a recommendation, but it cannot independently authorize the exception.

Engineering approach

1. Extract claim details and supporting documents.
2. Validate the extracted fields.
3. Retrieve the applicable reimbursement policy.
4. Identify missing documentation and potential exceptions.
5. Determine whether the case requires human approval.
6. Prepare a review summary with supporting evidence.
7. Route the case to an authorized reviewer.
8. Allow the reviewer to request clarification, reject, or approve within their authority.
9. Block financial execution until required approvals are recorded.
10. Record the outcome and relevant audit information.
11. Evaluate whether the escalation and review worked correctly.
12. Use findings to improve the workflow.

Illustrative review summary

Claim: ₹75,000
Status: Human review required
Reasons: Missing receipts and requested policy exception
AI recommendation: Request additional evidence and authorized review
Permitted next action: Route to the designated reviewer

Industry insight: AI can prepare the information needed for a decision without being granted the authority to make that decision.

↑ Back to contents

10. Try It Yourself

Mini Project: Design a Human Approval Workflow

Objective: Create a simulated HITL workflow for an AI reimbursement assistant.

1. Define the assistant's permitted responsibilities.
2. Create a fictional expense reimbursement policy.
3. Prepare ten fictional claims.
4. Include routine, incomplete, exceptional, and high-impact cases.
5. Define explicit human-review triggers.
6. Use AI to extract information and propose next steps.
7. Route designated claims to a simulated review queue.
8. Prepare a reviewer interface or decision form.
9. Include approve, reject, request-information, and escalate options.
10. Simulate authorization checks before processing.
11. Record reviewer decisions and processing time.
12. Evaluate missed escalations and unnecessary reviews.
13. Revise the routing rules and retest.

Suggested Evaluation Measures

Measure	Question
Escalation recall	Were cases requiring review correctly identified?
Unnecessary review rate	How many routine cases were escalated unnecessarily?
Review turnaround time	How long did decisions take?
Unauthorized execution count	Did any restricted action execute without approval?
Reviewer correction rate	How often did reviewers change AI recommendations?
Decision traceability	Can the outcome and responsible reviewer be identified?

Reflection Questions

• Which decisions should remain with humans?
• When is human review unnecessary?
• How can automation bias affect reviewers?
• What happens if a reviewer approves an incorrect recommendation?
• How should the system respond when reviewers are unavailable?
• How can HITL improve both efficiency and accountability?

Success criterion: Produce a workflow design with review triggers, reviewer permissions, simulated approval controls, audit records, and measurable evaluation results.

↑ Back to contents

11. Connected Glossary Terms

• Prompt Engineering: Instructions can define when the model should request review.
• RAG: Retrieved evidence can support human decisions.
• Evals: Tests can measure escalation accuracy and review outcomes.
• Hallucination: Review can help identify unsupported claims.
• Grounding: Evidence supports informed human judgment.
• Guardrails: Authorization controls enforce mandatory approvals.
• Observability: Logs and traces record review and execution events.

Previous lessons:

1. Token
2. Embedding
3. Vector Database
4. RAG
5. Context Window
6. Prompt Engineering
7. Evals
8. Hallucination
9. Grounding
10. Guardrails
11. Observability

Next lesson: #013 — Tool Calling: Connecting AI Models to External Capabilities.

↑ Back to contents

12. Living Update

Editorial baseline: November 2026

Human oversight design continues to evolve as AI applications become more autonomous and gain access to tools and enterprise systems.

Important areas to monitor include:

• Risk-based review routing.
• Approval checkpoints for AI agents.
• Reviewer interface design and automation bias.
• Human oversight effectiveness evaluations.
• Auditability of AI-assisted decisions.
• Adaptive escalation workflows.
• Regulatory requirements for meaningful oversight.
• Balancing review capacity and system performance.

Living Glossary note: Update this entry as operational practices and regulatory requirements develop.

↑ Back to contents

13. Key Takeaway

Human-in-the-Loop places meaningful human judgment and authority at the points where they are needed in an AI workflow.

Effective oversight is not achieved merely by inserting an approval button. It requires the right reviewer, relevant evidence, genuine intervention capabilities, and enforceable authorization controls.

Think like an AI engineer: Can the human reviewer understand, challenge, and stop the AI's proposed action before consequences occur?

↑ Back to contents

14. References and Further Reading

• NIST — AI Risk Management Framework
• NIST — AI RMF Playbook and Knowledge Base
• European Commission — AI Act Service Desk
• ISO/IEC 42001 — AI Management Systems
• OWASP — Generative AI Security Project

↑ Back to contents

AI Engineering — Living Glossary

One Term. One Industry Connection. One Practical Insight. One Action.

AIYING — Accomplishing with AI

Chapter 2 | Episode #012 | HUMAN-IN-THE-LOOP

↑ Return to top

AI FLUENCY | AI AGENT ENGINEERING

AI Agent Human-in-the-Loop: When Should AI Agents Ask for Human Approval?

AI agents can plan, recommend, and execute actions. But should every action happen without human approval?

Discover how Human-in-the-Loop (HITL) keeps people involved in important AI decisions.

Understand → Apply → Build → Test

AIYING — Accomplishing with AI

🎬 Watch the Short Lesson

Watch how an AI agent pauses before a sensitive action and requests human approval.

Watch on YouTube Shorts ↗

1. What Is Human-in-the-Loop?

Human-in-the-Loop (HITL) is an approach where a human participates in an AI-supported process by reviewing, approving, rejecting, or correcting selected actions.

In AI agent engineering, HITL commonly means that an agent pauses before executing a sensitive tool action and waits for authorization.

Simple explanation: The AI agent can propose an action, but an authorized person decides whether that particular action should proceed.

Key Principle

AI Proposes → Human Decides

Human approval provides an additional control but does not replace authentication, authorization, or system-level security.

2. How Does Human-in-the-Loop Work?

🤖 AI Agent Proposes an Action

↓

⏸️ Runtime Pauses the Tool Execution

↓

👤 Human Reviews the Proposed Action

↓

✅ Approve or ❌ Reject

↓

⚙️ Execute Approved Action or Block Rejected Action

If the action is approved, the agent runtime may execute the pending tool call.

If rejected, that specific tool call should not execute. The agent can instead report the rejection or propose another permitted approach.

Important: An approval should apply to the reviewed action and arguments, not automatically authorize every future action.

3. Where Is HITL Useful?

• Customer Support: Supervisor approval before issuing certain refunds or changing customer accounts.
• Communications: Review an AI-generated email before sending it externally.
• Financial Operations: Human authorization before purchases or financial transactions.
• IT Operations: Review high-impact system changes before execution.
• Content Publishing: Editorial approval before publishing customer-facing information.

4. Industry Application: Customer Support

Illustrative scenario: An AI customer-support agent prepares an email about a customer's refund request.

Before sending the email, the agent pauses for supervisor approval.

The supervisor checks:

• Correct customer and recipient
• Refund amount and applicable policy
• Accuracy of the message
• Appropriateness of the proposed action

The supervisor can approve or reject the pending sending action.

Industry benefit: Human review helps organizations retain oversight of consequential customer-facing actions.

This is an illustrative industry scenario, not a documented deployment by a particular company. Production systems also require identity verification, access controls, policy enforcement, and audit records.

5. Build an Approval-Gated AI Email Agent

Let's build a small AI agent that prepares an email and requests approval before executing a simulated sending action.

Learning objective: Understand how to pause an AI agent, inspect a proposed tool call, and approve or reject its execution.

Prerequisites

• Python 3.10 or later
• OpenAI API access
• An API key configured securely
• OpenAI Agents SDK

Step 1: Install the SDK

pip install -U openai-agents

Step 2: Create the Agent

Save the following code as approval_email_agent.py.

This project simulates sending an email. It does not connect to Gmail, Outlook, or any actual email service.

import asyncio

from agents import Agent, Runner
from agents.decorators import tool


@tool(needs_approval=True)
async def send_email(
    recipient: str,
    subject: str,
    body: str
) -> str:
    """Simulate sending an email.

    This tool does not contact an
    actual email service.
    """

    print("DEMO EMAIL ACTION EXECUTED")
    print("To:", recipient)
    print("Subject:", subject)
    print("Body:", body)

    return (
        "Demo email action completed. "
        "No real email was sent."
    )


agent = Agent(
    name="Approval-Gated Email Assistant",

    instructions=(
        "Prepare and send demo emails "
        "when requested. Use the send_email "
        "tool for a demo sending action. "
        "Never claim a real email was sent."
    ),

    tools=[send_email],
)


async def main():

    result = await Runner.run(
        agent,
        (
            "Prepare and send a short "
            "meeting reminder to "
            "teammate@example.com "
            "for tomorrow at 10 AM. "
            "Use the demo email tool."
        ),
    )

    while result.interruptions:

        state = result.to_state()

        for request in result.interruptions:

            print(
                "PENDING TOOL:",
                request.tool_name
            )

            print(
                "ARGUMENTS:",
                request.arguments
            )

            decision = input(
                "Approve this DEMO action? "
                "(yes/no): "
            ).strip().lower()

            if decision == "yes":
                state.approve(request)
            else:
                state.reject(request)

        result = await Runner.run(
            agent,
            state
        )

    print(
        "Agent response:",
        result.final_output
    )


if __name__ == "__main__":
    asyncio.run(main())

Step 3: Run the Agent

Configure the OPENAI_API_KEY environment variable securely.

Then execute:

python approval_email_agent.py

Step 4: Review the Approval Request

When the agent requests the demo sending tool, the SDK should pause and expose the pending tool call for review.

The application displays the proposed recipient, subject, and message body as part of the tool arguments.

You can choose:

• yes: Approve the pending demo action.
• no: Reject the pending demo action.

The application then resumes the agent with the selected decision.

The example follows the OpenAI Agents SDK interruption and resume workflow. Exact model-generated wording may vary. Live API execution has not been independently verified for this lesson.

6. Test Your AI Agent

1. Approval Test: Choose "yes" and verify that the demo sending action executes only after approval.
2. Rejection Test: Start a new run, choose "no", and verify that the rejected action does not execute.
3. Argument Review: Inspect the recipient, subject, and message body before making a decision.
4. Approval Scope: Verify that a separate sensitive tool call requires its own approval.
5. Security Test: Confirm that human approval does not bypass identity, permission, or authorization requirements.

Engineering takeaway: A safe approval process must distinguish proposing an action from executing it.

It must also ensure that approvals are genuine, appropriately scoped, and recorded when required.

7. Review Questions

1. Why should an AI agent pause before executing certain actions?
2. What is the difference between proposing and executing an action?
3. Why is human approval insufficient without proper security controls?

8. Interactive Quiz

Test your understanding of Human-in-the-Loop AI agents with five questions.

9. Official References and Further Learning

• OpenAI Agents SDK — Human-in-the-Loop
• OpenAI Agents SDK — Python Documentation
• OpenAI — Human-in-the-Loop Example

What's Next?

AI Agent Evaluation

How can we test whether an AI agent is reliable, accurate, and safe?

Continue exploring practical AI Agent Engineering.

AIYING — Accomplishing with AI

https://dastikop.blogspot.com