Human-in-the-Loop · HITL
Human-in-the-Loop
What is Human-in-the-Loop?
- A design that builds human review and intervention points into AI processes
- The last line of defense against errors and hallucinations, and the basis for assigning accountability
- The higher the risk of a decision, the stronger the human involvement
Once you start designing HITL, the questions inside the organization change. Instead of 'How accurate is the AI?', the question becomes 'When it's wrong, who catches it, and at what point?' No model will ever reach 100% accuracy, but you can build a checkpoint that catches errors before they turn into real actions. The practical effect of HITL is to shift the discussion from a race for accuracy to the design of control points.
Human involvement usually comes in three layers: pre-execution control, where a person must approve before anything proceeds; human-on-the-loop (HOTL), where the system runs automatically and people spot-check logs afterward; and human-in-command (HIC), where people hold the authority to start or stop the system itself. In practice, these are mixed. For example, model confidence of 90% or higher is processed automatically, 70–90% goes to a review queue, and below 70% is rejected automatically. One caveat: having a person look at a screen does not make it HITL. It works as a control point only when the reviewer has both the authority to overturn the result and the time to judge.
The approval button is where things most often break down in practice. When 300 cases a day pile up on one reviewer, people lean toward simply following the AI's judgment (automation bias), and the audit log fills up with 'human approved' entries while actual review drops to virtually zero. If an incident occurs in this state, accountability falls on the individual who clicked last, and HITL becomes a fig leaf. Set a cap on review volume, require a one-line approval reason, and monitor the rejection rate. If the rejection rate stays at 0%, review is not working.
John KimInstructor John Kim, author of B2B Marketing Design and CEO of Nanum Consulting
Seolyeong ParkGenerative AI and AX training specialist with 17 years of hands-on experience and 200+ lectures, helping non-technical staff put AI to work right awayThe concept grew out of research on human intervention in control systems and machine learning, and became a standard term in AI governance as generative AI and agents spread.
Consider a hypothetical case. Company A, a retailer with 420 employees, introduced AI résumé screening for its second-half graduate hiring. There were 2,400 applicants, but only 3 people in HR to review résumés, and the screening window was just 2 weeks. In the first week, applicants began writing in to say they were being 'rejected without knowing the criteria,' and the HR head redesigned where humans step in.
- The head of HR scaled back the AI's authority. The AI would screen out only fact-based shortfalls, such as years of experience or required certifications, and rejecting anyone on a fit score alone was prohibited.
- For the first three weeks, the AI and human reviewers ran in parallel. Two recruiters re-reviewed all 310 applications the AI had rejected and found 17 wrongful rejections, caused by differences in how work experience was formatted.
- In the review meeting, one recruiter said, "This is a résumé that would have been rejected if no one had looked at it." The team lead then changed the rule so that scores of 60–75 go to 'hold' and are all passed to a person.
- The approval screen was reworked too. A one-line reason is now required for every pass or reject click, and each reviewer is capped at 120 cases a day to prevent mechanical click-through.
- Three months later, wrongful rejections fell from 17 to 2, and résumé screening time dropped from 11 days to 4. Applicant objections fell from 23 a month to 4.