The Review Step Works Until the Month Ends
We put a person in the loop and called the problem solved. Then we looked at approval rates by day of month and found that the check we were relying on stopped functioning during exactly the period when volume and pressure were highest.
The Number That Told Us
Extracted records that fail a validation rule go to a review queue where a person confirms or corrects them. For most of the month, reviewers changed something in about eleven percent of the records they opened.
In the last four working days of the month, that fell to under three percent, while the volume in the queue roughly doubled. The records were not better in that period. We checked a sample afterwards, and the error rate in what had been approved was about four times the rate in the rest of the month.
What We Had Assumed
That a human check is a constant. We had designed the pipeline around it: anything uncertain goes to review, and review catches it. That assumption is implicit in almost every human-in-the-loop diagram, including the ones we had drawn for customers.
It is not a constant. It is a component with a throughput, a fatigue curve and a dependence on how much time the reviewer believes they have, and none of those appear in the diagram. We had modelled the people as infallible and the queue as free.
The Research That Fits
Haduong and colleagues published a study that October on how performance pressure changes the way people work with AI assistance, examining whether pressure makes people more or less inclined to rely on the system rather than on their own judgement.
The value for us was not a specific number but the reframing. Reliance on an assistant is not a fixed property of a person, it varies with the conditions they are working under, which means a review step designed for calm conditions is not the review step that runs at month end.
| What we changed | Why |
|---|---|
| Cap the queue per reviewer per hour | Beyond it, quality falls faster than volume |
| Show one record at a time, no list | A list invites approving in bulk |
| Sample approved records for audit | The only way to see the failure at all |
| Hold non-urgent records for later | Move work out of the peak, not through it |
What We Changed in the Queue
Not the people. We capped how many records a reviewer sees per hour, and beyond the cap the remainder waits or routes to someone else. That sounds like it slows the month end down, and it does, by an amount we can state.
We also removed the list view. Reviewers now see one record with its source document, and the next only after deciding. The list had made it possible to approve fourteen records in a minute, which is not review, and removing it was the single change with the largest effect.
The Audit That Makes It Visible
Two percent of approved records are re-checked by a second person, sampled with a bias toward high-volume periods. That is the only mechanism that would have shown us the problem, and we did not have it, which is why the failure ran for months before anyone noticed.
The audit rate is deliberately small and deliberately not secret. Reviewers know it exists and know it is not about them individually, and we report the aggregate rather than per-person figures, because the alternative produces careful-looking behaviour rather than careful behaviour.
What This Says About the Design
If a human check is load-bearing, it needs the same treatment as any other component: a stated capacity, monitoring, and a defined behaviour when it is exceeded. Ours now has all three, and the defined behaviour is that work waits rather than passing through unchecked.
That principle has an uncomfortable consequence. A pipeline whose safety depends on review cannot process more than its reviewers can handle, so the throughput of the automated part is bounded by the human part. We say that during design now, because customers reasonably assume automation removes the bound.
The Alternative We Chose Against
We could have reduced what goes to review, by loosening the validation rules that route records there. That would have cut the queue at the peak and it is the wrong direction: fewer records reviewed is not the same as fewer records needing review.
Instead we reduced what needs review by fixing the extraction failures the queue was catching, which took longer and moved the volume down permanently. The queue at month end is now about forty percent of what it was, from the same document volume.
What We Do Not Claim
We do not claim our caps are correct. They are where the audit sample stopped showing a difference, measured on one team doing one kind of review, and a different task would have a different curve.
We also do not claim this is a criticism of reviewers. They were doing exactly what the situation asked of them, and the failure was ours: we built a process that quietly required people to work at a constant rate regardless of load, and then treated the result as a quality problem.