Human in the loop: designing AI workflows your reviewers will trust

There is a moment that decides whether an automation survives its first month. A senior reviewer opens the output, looks at a figure, and asks where it came from. If the answer takes thirty seconds, the workflow is on its way to becoming normal. If the answer is "the system produced it", the reviewer re-does the work by hand — and quietly keeps doing that forever.

"Human in the loop" is often used as reassurance: a person is involved somewhere, so the risk is managed. That is not a design. A person being nominally responsible for checking output they cannot practically check is worse than no automation at all, because it manufactures the appearance of a control.

1. Decide what the human is actually deciding

The first question is not "where does a person review this?" but "what specific decision does the person make, and what do they need in front of them to make it?"

"Review the workpaper" is not a decision. "Confirm that these four flagged figures match their source documents, and determine whether the vehicle expense is deductible" is. The second version is bounded, it is checkable, and the reviewer knows when they are finished. The first version expands to fill whatever time is available and leaves the reviewer uncertain whether they have done enough.

2. Make provenance a first-class output

Every value an automation produces should carry its origin: which document, which page, which field. Not in a log file somewhere, but visible at the point of review, one click away from the number itself.

This is the single highest-leverage thing you can build, and it is routinely skipped because it does not improve accuracy. It improves verifiability, which is what actually determines whether the time saving is real. An output that is 99% accurate but unverifiable saves nobody anything.

Accuracy determines whether the automation is any good. Verifiability determines whether anyone gets to benefit from it.

3. Put the gate where the cost of being wrong changes

Review gates should sit immediately before the point at which an error becomes expensive to reverse — not distributed evenly through the process, and not all bunched at the end.

In a return preparation pipeline, a mis-read figure is cheap right up until the moment it flows into a lodgement. So the meaningful gate is before lodgement, not after each extraction step. In a client correspondence workflow, the cost changes the moment something is sent, so the gate is before send — and everything upstream of it can run unattended.

Gates in the wrong place are the main reason teams describe automation as "more work than doing it myself". They are usually not wrong about that — they are describing a workflow that asks them to approve things whose correctness cannot yet be assessed.

4. Design the disagreement path before you need it

What happens when the reviewer disagrees with the output? In a lot of implementations the honest answer is: they fix it in the file and move on, and the system never learns anything.

That is a missed opportunity twice over. The correction is the most valuable signal you will ever get about where the automation is weak, and capturing it costs almost nothing if you design for it upfront. At minimum, record what was changed, from what to what, and let someone look at the pattern monthly. Corrections that cluster are telling you about a systematic gap, not a one-off.

• Capture the override — the original value, the corrected value, and the reviewer.
• Cluster the overrides — the same field being corrected repeatedly is a defect, not noise.
• Close the loop — tell the team what changed as a result, or they will stop bothering to correct properly.

5. Let the system say "I don't know"

A workflow that always produces an answer is a workflow that will confidently produce a wrong one. Every automation needs a legitimate path to abstain — to mark an item as unresolved and route it to a person without pretending.

This sounds obvious and is surprisingly rare, because abstention looks like failure in a demo. It is the opposite. A pipeline that resolves 85% of items cleanly and escalates 15% with a clear reason is more useful, and far more trusted, than one that resolves 100% and is silently wrong in 4% of cases.

---

Why this is a competitive question, not just a risk one

It is tempting to file all of this under compliance — something you do to keep the professional standards body and the PI insurer comfortable. That undersells it.

Firms that get the review design right can push automation into progressively higher-stakes work, because they have a mechanism for knowing when to trust it. Firms that do not stay stuck automating only the trivial, because every attempt to go further runs into a reviewer who reasonably refuses to sign. The ceiling on how much of your practice you can automate is set by how well you designed the human's role, not by how good the underlying technology is.

If you are still deciding which workflow to take on first, the companion piece on where AI actually saves time covers how to choose — and the guide to preparing individual tax returns with AI shows these principles applied to a real pipeline.