Human in the Loop testing: Where People Still Belong in Agentic Testing

The objection comes up in nearly every conversation about agentic testing, usually phrased carefully, and it is a reasonable one: *we are not comfortable with AI making decisions without a person checking them.*

The instinct behind that is sound. Autonomous systems acting on financial data without oversight is a genuinely bad idea, and anyone dismissing the concern is not worth listening to.

But the conversation usually stops at “we’ll keep a human in the loop,” and that phrase does far less work than people think. It describes at least four different arrangements with very different costs, very different assurance, and, this is the part that matters, one of them is actively worse than having no oversight at all.

Everyone agrees a human should be in the loop. Almost nobody specifies which loop, and that is where the value is decided.

This piece covers the four arrangements, why the wrong one produces false assurance, where human judgment is genuinely irreplaceable in testing, and how to design an escalation boundary that puts people where their judgment matters rather than where the volume is.

Four Arrangements, Not One

The phrase covers four distinct designs. They are not interchangeable and they do not cost the same.

Most organizations reach instinctively for the first, because it feels like the most control. It is also the one that destroys the value of automation entirely, an agent processing four thousand transactions a day gated behind human approval is a very expensive way to run a queue.

The two that actually work at scale are the bottom two, and they are the ones least discussed. **In the design** is where nearly all the leverage sits: a person defining what correct means, what the boundaries are, and what must escalate does that work once and it governs every subsequent transaction. **In the exception** scales with how often the agent encounters something genuinely ambiguous rather than with how much work it does.

Human in the design scales to infinity. Human in the loop scales to about forty transactions before someone stops reading.

Why the Wrong Loop Is Worse Than No Loop

Here is the part that deserves more attention than it gets.

Put a person in a blocking approval role over a high-volume agent and you get a predictable sequence. For the first day or two they read each item carefully. Then the volume asserts itself, the agent turns out to be right almost every time, and approval becomes a rhythm rather than a judgment. By the second week, clicking approve is a motor action.

This is not a failure of diligence. It is what happens to any human asked to review a high volume of items that are nearly always correct, the base rate teaches you to expect correctness, and attention follows expectation.

The AI agent governance for ERP consequence is the serious part. You now have an audit trail stating that a named individual reviewed and approved four thousand transactions. That record is worse than an honest one saying no human reviewed them, for two reasons: it asserts assurance that does not exist, and it places accountability on a person who was not given a realistic opportunity to exercise it.

Rubber-stamped approval does not produce oversight. It produces documentation that oversight happened.

Which is why the design question is not “do we have a human in the loop” but “is the human we placed there able to do the thing we are claiming they do.” If the honest answer is no, the arrangement should change, not the paperwork.

The Automation Paradox

There is a second-order problem that shows up later and is worth designing against early.

Lisanne Bainbridge described this in the early 1980s as one of the ironies of automation: the more reliable an automated system becomes, the less practiced the human overseeing it becomes at the underlying task, and therefore the less capable of handling the cases the automation cannot. Automation takes the routine work, which is exactly the work through which people maintain their skill.

Applied here: an agent that codes invoices correctly 99% of the time produces a team that has not manually coded an invoice in eight months. When the 1% arrives, the genuinely ambiguous case that needs judgment, the person receiving it has less context than they would have had before the agent existed.

This does not argue against automation. It argues for two specific things: keeping humans engaged with the interesting cases rather than only the exceptions that are also boring, and treating the ability to judge agent output as a skill that needs deliberate maintenance rather than one that persists automatically.

Practically, that tends to mean periodic review of a stratified sample weighted toward unusual cases, not because agent observability drift well, but because it keeps the reviewer’s judgment calibrated.

Where Human Judgment Is Genuinely Irreplaceable

Five areas, and none of them are about volume.

• Defining what correct means. An agent validates against rules. Someone has to decide what the rules should be, and that decision encodes business intent, regulatory interpretation and risk appetite. Automation can verify conformance to intent; it cannot supply the intent.
• Judging whether a rule is right, not whether it was followed. If your account derivation logic assigns costs to the wrong cost centre by design, every assertion passes and the reporting is still wrong. Only a person with domain context catches that, and only by looking at outcomes rather than at test results.
• Novel situations with no precedent. Agents reason from patterns. A genuinely unprecedented scenario, a new regulatory requirement, an acquisition creating an entity structure the system has never handled, has no pattern to reason from. This is where escalation should be automatic rather than threshold-based.
• Risk appetite. How much autonomy an agent should have is a business decision informed by evidence, not derived from it. Testing tells you how reliably an agent performs within a scope. It cannot tell you how wide that scope should be, and the answer depends on consequences the system cannot weigh.
• Accountability. This one is not a capability gap and cannot be closed by better agents. Regulatory frameworks require a named accountable person SOX requirement for AI Agents, and an agent cannot be held accountable in any sense that matters, legally, professionally or organizationally. Someone owns the outcome regardless of who performed the action.

Designing the Escalation Boundary

If human-in-the-exception is the arrangement that works, the whole design question becomes where the boundary sits. Four principles that hold up in practice.

Escalate on ambiguity, not on value. Value thresholds are the common default and a crude proxy. A large, entirely routine transaction needs less judgment than a small one where two treatments are both defensible. Where the agent has low confidence or the rules genuinely conflict is where a person adds something.

Escalate on novelty, automatically. Any scenario the agent has not encountered a close analogue for should route to a human by default, regardless of value. This is the case where pattern-based reasoning is weakest and the case most likely to be genuinely interesting.

Make escalation a first-class outcome, not a failure. If escalating counts against an agent’s success metrics, the pressure is to escalate less, which is precisely the wrong incentive. A falling escalation rate should prompt investigation rather than celebration, because it can mean the agent has stopped being cautious.

Give the human the reasoning, not just the item. An escalation arriving as “review this transaction” wastes the human’s time reconstructing context. An escalation arriving with what the agent considered, why it was uncertain, and which rules were in tension lets the person do the judgment work rather than the archaeology.

That last point determines whether human-in-the-exception is efficient or merely relocated work. A well-formed escalation takes a couple of minutes; a bare one takes twenty.

How This Should Change Over Time

The right arrangement at month one is not the right one at month eighteen, and treating it as fixed is a mistake in either direction.

The last row is the one most often skipped. A model update, a scope expansion, a change to the business rules the agent operates under, each invalidates the evidence that justified the current level of autonomy. Stepping back temporarily is cheap; discovering after the fact that autonomy outran validation is not.

How Sofy Fits Into Your Testing Strategy

Sofy’s position on this is that continuous validation ERP test agent what makes the good arrangements viable.

Human-in-the-exception only works if you can trust that the non-exceptions were correct, and that trust has to come from somewhere. Validating every agent-driven outcome at the data layer, rather than sampling, is what allows a person to focus on escalations without the unexamined remainder being an act of faith.

It also supplies what the escalation design needs. Because validation runs against defined business rules, a case where rules conflict or where an outcome falls outside expected tolerance is identifiable rather than something a human has to discover by reading. That is the difference between escalating an item and escalating an item with its reasoning attached.

And the maturity progression above requires evidence at each step, agreement rates in shadow mode the five validation dimensions, override frequency, escalation stability. Those are measurements rather than impressions, and producing them is a validation function.

What We Don’t Know Yet?

Being straightforward about the state of the practice, because this is an area where confident claims should be treated with suspicion.

There is no established answer to how much autonomy is appropriate for a given risk level. The frameworks that exist are extensions of pre-agentic control thinking, and the empirical basis, what actually goes wrong, how often, at what autonomy level, is thin because most deployments are less than two years old.

Nor is there good evidence on how quickly human judgment degrades when agents handle routine work in this specific context. The automation paradox is well established in aviation and process control; how strongly it applies to financial process oversight is a reasonable inference rather than a measured finding.

The practical implication is to keep autonomy narrower than the technology permits, and to widen it on evidence you gathered rather than on a vendor’s confidence, including ours. Anyone telling you the correct autonomy level for your agents is telling you about their product rather than about your risk.

Frequently Asked Questions

What does human in the loop testing actually mean for AI agents?

It covers four distinct arrangements: human in the loop (approving each action before execution), on the loop (monitoring with the ability to intervene), in the design (defining what correct means and where boundaries sit), and in the exception (handling only escalations). They differ substantially in cost and assurance, the first collapses throughput to human speed, while the last two scale.

Why is blanket human approval a bad idea?

Because it produces rubber-stamping rather than review. Asked to approve a high volume of items that are nearly always correct, attention follows the base rate and approval becomes a motor action within days. The result is an audit trail asserting that a named person reviewed thousands of transactions, which is worse than an honest record of no review, because it claims assurance that does not exist and places accountability on someone not given a realistic chance to exercise it.

Where do humans genuinely add value in agentic testing?

Five places, none of them volume-related: defining what correct means (automation verifies conformance to intent but cannot supply intent), judging whether a rule is right rather than whether it was followed, handling genuinely novel scenarios with no precedent to reason from, setting risk appetite and autonomy scope, and carrying accountability, which is a legal and organizational requirement no agent capability changes.

How should escalation from an agent to a human be designed?

Escalate on ambiguity rather than transaction value, since a large routine transaction needs less judgment than a small contested one. Escalate on novelty automatically. Treat escalation as a successful outcome rather than a failure, or the incentive becomes escalating less. And include the agent’s reasoning, what it considered, where it was uncertain, which rules were in tension, so the human does judgment work rather than reconstructing context.

Should human oversight decrease as agents prove reliable?

Gradually, on evidence, and it should step back after any change. A model update, scope expansion or business rule change invalidates the evidence that justified the current autonomy level, so the sensible response is to return to a more supervised arrangement until re-validated. The common failure is treating autonomy as a ratchet that only loosens.

The Bottom Line

The objection that started this is right. Autonomous agents acting on financial processes without human oversight is not a defensible position, and the answer is not to argue people out of the concern.

The answer is to be specific about which loop. Blanket approval feels like the safest choice and is the one that reliably degrades into documentation of a review that did not happen. The arrangements that actually hold are humans in the design, defining what correct means, once, governing everything after, and humans in the exception, handling what genuinely requires judgment.

Put people where their judgment is the scarce resource, not where the volume is. And keep autonomy narrower than the technology allows until you have your own evidence that it should widen.