An Assistant in Front of the Queue Changes the Arithmetic
The assistant was doing well by every measure we had. Support throughput had not improved, and the reason turned out to be in an operations research textbook rather than in anything we could tune.
Good Numbers, No Effect
Forty percent of incoming requests resolved without a person. Answer quality on the labelled set at ninety-three percent. User satisfaction on the resolved cases higher than for human handling, which surprised everyone.
Support team throughput: unchanged. Average time to resolution across all requests: slightly worse. Six months of work had produced a system that was working and an outcome that was not.
The Model That Explains It
Dada and colleagues published work in 2026 on customer service operations with a gatekeeper framework, modelling the first stage that either resolves a request or refers it onward, and the conditions under which such a stage helps or hurts overall performance.
The mechanism is simple once stated. A request the gatekeeper resolves costs one cheap handling. A request it refers correctly costs one cheap handling plus one expensive one. A request it refers to the wrong place costs a cheap handling, a wrong expensive handling, and then the right one.
| Outcome | Cost in handlings |
|---|---|
| Resolved by the assistant | One cheap. The saving |
| Referred correctly | Cheap plus expensive. Small overhead |
| Referred to the wrong team | Cheap plus two expensive. The problem |
| Resolved wrongly, user returns | Worst case. Counts twice and erodes trust |
Where Our Six Percent Went
Misroutes are not evenly costly. Sending a billing question to technical support costs a triage and a forward, perhaps ten minutes. Sending a warranty claim to the wrong region costs a day because it enters a different queue with a different service level.
Our six percent was concentrated in the expensive kind, because those are the requests whose correct destination depends on a distinction the assistant was not equipped to make. The average misroute rate was fine and the weighted cost was not.
What We Changed
Not the model. We reduced the number of destinations it may route to from eleven to four, with the four chosen so that a wrong choice among them is cheap to correct. Anything that would have gone to one of the other seven now goes to human triage.
Misroute rate rose to nine percent and the weighted cost fell by more than half, because the remaining misroutes are all in the cheap category. That is the trade the queue model makes visible and that an accuracy metric hides completely.
The Second Change: Refusing to Resolve
The assistant now refers rather than resolves when a request mentions a deadline, a complaint or a legal term, regardless of whether it could answer. Those categories are where a wrong resolution produces a returning user, and a returning user costs two handlings and some goodwill.
Resolution rate fell from forty percent to thirty-four. Throughput improved for the first time, because the six percentage points we gave up were the ones most likely to come back.
The Metric We Were Missing
Cost-weighted outcome per request, rather than resolution rate and accuracy separately. Each outcome carries a handling cost from the support team's own figures, and the assistant is judged on the total rather than on how often it answers.
It is a worse metric for a dashboard and the only one that moved when the system got better. Resolution rate and satisfaction had both looked good throughout the period when the system was making things marginally worse.
What This Means for Scoping These Projects
The support process has to be modelled before the assistant is designed, because the destinations and their costs determine what the assistant should be allowed to do. We now ask for the handling cost per queue in the first workshop, which nobody has ever had to hand and everybody can produce in a week.
Without it, the project optimises what is measurable about the model rather than what the customer is paying to improve, which is how a system with good numbers produces no effect.
What We Do Not Claim
We do not claim resolution rate is a bad metric. It is the right metric for the assistant as a component, and our point is that the component metric and the outcome metric came apart and we had only been watching the first one.
We also do not claim our four destinations generalise. That number came from this customer's cost structure, and an organisation whose queues have similar handling costs would have no reason to restrict routing at all.