Aiodic

Back to the table

54 Intermediate Hi Human in the Loop Safety & Ethics

Human in the Loop

People review or approve AI actions.

What it is

Human in the loop means a person reviews, corrects or approves what an AI does before it counts. The AI drafts the email and you press send. The AI suggests which invoices look wrong and an accountant decides. An AI agent proposes deleting 300 old files and waits until someone clicks "approve". The AI does the fast, tireless part of the work. A human keeps the decisions that matter.

The phrase comes from engineering, where a "loop" is the cycle of sensing, deciding and acting that any control system runs. Put a person inside that cycle and nothing important happens without them. Take the person out and the system runs on its own. Between those two ends there are three positions worth knowing by name:

None of these is good or bad on its own. The skill is choosing the right position for each step of a job. A useful rule is to put approval gates in front of three kinds of action:

Everything else, such as drafting, summarising, sorting and suggesting, can usually run with light checking or none, because a mistake costs you a few seconds.

You'll also meet the phrase in a second, related sense inside machine learning. There, "human in the loop" describes people labelling examples or rating a model's answers so it learns from them, as in supervised learning and RLHF. This article is mostly about the first meaning: people checking what AI does at the moment it does it. The two share one idea, though. A human's judgement is the reference point the machine is measured against.

Why it matters

AI makes confident mistakes. Language models sometimes invent facts (the hallucination problem), misread instructions, and carry the bias of their training data. An agent can do all of that and then act on it. A person at the right checkpoint catches the error before it becomes an email to a client or a deleted folder.

Somebody has to be answerable. When something goes wrong, "the computer did it" is not an answer a customer, a court or a regulator accepts. Keeping a named person in key decisions keeps responsibility where it can actually be exercised. Law is moving the same way. Article 14 of the EU AI Act requires that high-risk AI systems, such as those used in hiring, credit and access to public services, be designed so that people can oversee them effectively while they are in use. It asks that the people doing the oversight can decide not to use the system's output, and can halt it with a "stop" button or a similar procedure. For remote biometric identification it goes further and requires that a match be separately confirmed by at least two qualified people before anyone acts on it. At the time of writing, a 2026 amendment has moved the start date for most of these high-risk rules to December 2027, so check the current timetable if it affects you.

But a human in the loop is not automatically a safeguard. This is the part most people miss, and the same Article 14 names it. It asks that overseers stay aware of automation bias, which the law describes as the tendency to rely, or over-rely, on the system's output. Researchers have studied this for decades. In a 2010 review in the journal Human Factors, Raja Parasuraman and Dietrich Manzey pulled together the evidence on two related failures. Automation complacency is when people stop checking a system that is usually right, especially when they are busy with other tasks. Automation bias is when people follow an automated suggestion even when it is wrong (a commission error), or miss a problem because the system didn't flag it (an omission error). Their review found both in experts as well as beginners, found that complacency could not be overcome with simple practice, and found that training or instructions did not prevent automation bias.

The effect shows up with modern AI too. In a study published in the journal Radiology in May 2023, Thomas Dratsch and colleagues had 27 radiologists read 50 mammograms with suggestions from what they were told was an AI system. Some suggestions were deliberately wrong. When the suggestion was right, readers at every experience level got about 80 percent of cases correct. When it was wrong, inexperienced readers fell to about 20 percent, and even the very experienced fell to about 46 percent. The human was in the loop. The loop still failed.

Other articles in this table show the same pattern in different settings. In the Boston Consulting Group experiment described under iterative refinement, consultants given AI on a task it handled badly did worse than those without it, largely because they accepted its polished answer. Under bias, the 2020 wrongful arrest of Robert Williams in Detroit followed a facial recognition match that people treated as stronger evidence than it was.

Reviewing is tiring, and tired reviewers approve. If a person must approve 200 near-identical suggestions a day, and 199 are fine, they will soon approve the 200th without reading it. This is called rubber-stamping, and it turns review into a ritual. Too many approval requests produce the same result as none, only slower.

A stop button has to actually stop. In February 2026, as described under function calling, a researcher at Meta told an AI agent not to act on her inbox without approval, and it began deleting mail anyway after her instruction was lost; stopping it from her phone didn't work. An instruction inside the conversation is not the same as a control built into the system.

The researcher Madeleine Clare Elish gave a name to a final danger in a 2019 paper: the "moral crumple zone". When an automated system fails, blame tends to land on the nearest person, even if they had little real control. A human in the loop who can't see why the AI decided something, has no time to check and has no power to say no isn't a safeguard. They're a place for the blame to go.

Real-world examples

The Post Office and the Horizon computer system (1999 to today)

From 1999, the UK Post Office rolled out Horizon, an accounting system built by Fujitsu, to its branches. Branch operators, known as sub-postmasters, began reporting shortfalls on their screens that they could not explain. The Post Office treated the numbers as correct and the people as the problem. Hundreds were prosecuted for theft, fraud and false accounting, many lost their businesses and savings, and some went to prison. A 1999 change in the law of England and Wales, which let courts presume a computer was working properly unless shown otherwise, made those prosecutions easier. The Post Office had supported that change in the consultation beforehand.

Horizon is not AI, but it is the clearest modern case of an institution trusting a computer over the people who used it. On 16 December 2019, in a High Court case brought by 555 claimants led by Alan Bates, Mr Justice Fraser found that the system contained bugs, errors and defects capable of causing shortfalls in branch accounts. On 23 April 2021 the Court of Appeal quashed the convictions of 39 former sub-postmasters and staff. Parliament then passed the Post Office (Horizon System) Offences Act 2024, which received Royal Assent on 24 May 2024 and quashed qualifying convictions in England, Wales and Northern Ireland in one go. A public inquiry chaired by Sir Wyn Williams published the first volume of its final report, on the human impact and compensation, on 8 July 2025.

The lesson for AI is uncomfortable. There were humans in the loop throughout: investigators, managers, lawyers and judges. The problem was that each one deferred to the machine. A human check only works if the human is allowed, and expected, to believe the person over the printout.

The Uber test car in Tempe, Arizona (March 2018)

On the night of 18 March 2018, an Uber test vehicle driving itself in Tempe, Arizona, struck and killed Elaine Herzberg, 49, as she crossed the road pushing a bicycle. It is widely reported as the first pedestrian death involving a fully self-driving car. A safety driver sat behind the wheel. Her job was to be the human on the loop: watch the road and take over if the system got it wrong.

The US National Transportation Safety Board (NTSB) found that the car's system detected her 5.6 seconds before impact but never correctly identified her or predicted her path. It concluded that the probable cause was the safety driver's failure to monitor the road because she was visually distracted by her personal phone throughout the trip. It also named contributing causes at the company, including ineffective oversight of vehicle operators and a lack of adequate ways to deal with operators' "automation complacency", which it traced to an inadequate safety culture. Reporting on the board's November 2019 meeting noted that Uber had moved from two operators per car to one in late 2017. In July 2023 the safety driver pleaded guilty to endangerment and was sentenced to three years of supervised probation.

The case shows how hard "on the loop" really is. Asking a person to stare at a road for hours while a machine drives, and to react within a second or two when it fails, is asking for exactly the attention that Parasuraman and Manzey showed people struggle to give. The NTSB put much of the responsibility on the company because designing for that weakness was the company's job.

The Dutch childcare benefits scandal (2013 to 2021)

In 2013 the Dutch tax authority adopted a "risk classification model" to spot incorrect or fraudulent claims for childcare benefits. According to Amnesty International's October 2021 report "Xenophobic Machines", the model scored each application, and those with the highest scores were sent to a civil servant for manual review. Whether the applicant held Dutch citizenship was one of the factors in the score. The civil servant could label the claim fraudulent, with severe consequences: families were ordered to repay benefits that could reach tens of thousands of euros.

On paper this was a human in the loop. In practice, Amnesty found, the civil servant was given no information about why the system had flagged an application. The reviewer could only react to the flag, not question it. A parliamentary inquiry called the treatment of parents an "unprecedented injustice" in December 2020, and on 15 January 2021 Prime Minister Mark Rutte's government resigned over the scandal. In December 2021 the Dutch Data Protection Authority fined the tax authority 2.75 million euros for processing applicants' nationality data in an unlawful and discriminatory way, including its use as an indicator in the risk system.

The lesson is about what a reviewer needs. Someone who sees a score but not the reasons, has targets to meet and faces no consequence for agreeing with the machine will agree with the machine.

An everyday case: the online shop's refund agent

Imagine Dana, who runs a small online shop and sets up an AI assistant to handle customer emails. At first she has it draft every reply for her to approve. Within two weeks she is clicking "send" on forty drafts a day without really reading them, because they are nearly always fine. Then one draft promises a full refund on an item that is outside her return window, and she only notices when the customer quotes it back to her.

She redesigns the loop. Routine answers about delivery times go out automatically, and she reads a random five each Friday. Anything that mentions money, a refund or a complaint waits for her approval, with the customer's order history shown beside the draft. She now approves six emails a day instead of forty, and she actually reads all six.

Try it yourself

You need one task you already do with an AI tool, a piece of paper or a document, and about thirty minutes. Any chatbot, such as ChatGPT, Claude or Gemini, is fine for the prompts below. Features and plans change, so check what your own tool allows.

  1. Pick one AI-assisted workflow. Something you really do: answering customer emails, scheduling social posts, drafting invoices, summarising meetings, sorting job applications.
  2. List every step. Write down each thing that happens from start to finish, including the steps a tool does for you. Five to ten steps is typical.
  3. Mark each step Auto, Review or Approve. Auto means the AI does it and nobody checks at the time. Review means a person looks at a sample, or checks after the fact. Approve means nothing happens until a person says yes. Use the three questions from this article: is it irreversible, does it reach someone outside, and does it cost money, reputation or someone's rights? Any yes means Approve.
  4. Count your approvals. If you have more than a handful per day, you will rubber-stamp them. Move the low-risk ones to Review with sampling, so the approvals that remain get real attention.
  5. Write a three-line review checklist for your most important Approve step. Each line should be a yes-or-no question you can answer in under a minute, such as "Are all names, numbers and dates correct against the source?", "Does it promise anything I haven't agreed to?" and "Would I be comfortable if this were forwarded to my biggest client?"
  6. Check you can stop it. For any tool that acts on its own, find out how to pause or disconnect it, and try it once. Don't rely on telling the AI to stop in the chat.
  7. Set a sampling habit. For Review steps, pick a rhythm, such as five random items every Friday, and put it in your calendar.

A prompt to help map your workflow:

I use AI for this task: [describe the task in two or three sentences].
List each step of the workflow from start to finish, including steps
the tool does automatically. For each step, say whether it is
irreversible, whether it reaches anyone outside my business, and
whether a mistake would cost money, reputation or someone's rights.
Then suggest Auto, Review or Approve for each step, with a one-line reason.
Don't make up steps I haven't mentioned; ask me if something is unclear.

A prompt to draft your checklist:

Here is a step where I approve AI output before it goes out:
[describe the step and paste one real example of the output].
Write a three-line review checklist for it. Each line must be a
yes-or-no question I can answer in under a minute, and together they
should catch the mistakes most likely to cause real harm here.

Treat what the chatbot suggests as a first draft. You know your business and your customers; it doesn't.

Common mistakes and misunderstandings

Thinking "a human approves it" means it's safe. Approval is only as good as the attention behind it. If the reviewer can't see the reasons, has no time or feels unable to say no, the approval is a formality. The Post Office and Dutch cases both had humans in the process.

Approving too much. Asking for approval on every small step feels careful, but it trains people to click "yes" without reading. Save approvals for irreversible, external and costly actions, and use sampling for the rest.

Putting the checkpoint at the end only. Checking a finished 40-page report is harder than checking the outline and the sources along the way. Place checks where mistakes are cheapest to catch.

Hiding the evidence from the reviewer. Show the reviewer what the AI saw and why it suggested what it did: the source document, the customer's history, the factors behind a score. A bare "flagged" or "approved" invites blind agreement.

Relying on an instruction instead of a control. "Ask me before you delete anything" in a prompt can be forgotten, misread or lost when a long conversation is summarised. Real approval gates live in the tool's settings and permissions, where the AI can't skip them. See guardrails for the wider set of controls.

Believing training alone fixes automation bias. Research on automation bias found that instructions and training don't reliably prevent it. Design helps more: ask reviewers to form their own view before seeing the AI's, rotate who reviews, and keep the number of approvals low.

Making the human a crumple zone. If a person carries the blame but has no real power to stop or change what the AI does, the design is unfair and unsafe. Give reviewers clear stop rights, the time to use them and backing when they do.

Assuming out of the loop is always wrong. Fully automatic is fine for tasks that are low-stakes and easy to reverse. Forcing a person to approve spam filtering wastes the attention you need for decisions that matter.

Setting it once and forgetting it. As a tool proves reliable, you may move a step from Approve to Review. As it takes on bigger jobs, you may need to move steps back. Revisit your map whenever the tool, the task or the stakes change.

Related elements