What We Refuse to Automate
A customer asked us to automate a step that takes their staff about forty seconds. We said no, and the reason is the same one that makes assistant benchmarks embarrassing to read.
The Forty-Second Task
The step was deciding whether an incoming complaint should go to the technical queue or the commercial one. Staff did it in seconds, from the text alone, apparently without thinking. It looked like the most automatable thing in the process.
Watching people do it, the routing turned out to draw on things not present in the text: which customer this is, what happened on their last order, whether the phrasing sounded like an escalation from someone usually patient. None of that lives in a system an assistant can query.
Why Short Tasks Mislead
Duration is a bad proxy for difficulty, and it misleads in a specific direction: the tasks people complete fastest are often the ones drawing most heavily on context they do not notice using.
Mialon and colleagues built a benchmark around exactly this asymmetry, assembling questions that are conceptually simple for people yet hard for assistants. We read it less as a scoreboard than as a description of the gap we keep meeting in scoping conversations.
The Three Tests We Run
First, can the decision be made from data that exists in a system? If it depends on knowing the customer, and that knowledge is in someone's head, automation will approximate it and be confidently wrong at the edges.
Second, is the failure visible? A misrouted complaint surfaces within a day. A subtly misjudged one sits in the right queue with the wrong priority and nobody notices. Third, what does being wrong cost? We automate freely where errors are cheap and visible, and reluctantly where they are neither.
| We automate | We leave with a person |
|---|---|
| Decision follows from available data | Decision needs unrecorded context |
| Errors surface quickly | Errors are silent or delayed |
| Being wrong is recoverable | Being wrong reaches a customer or a regulator |
| High volume, stable shape | Rare, varied, judgement-heavy |
What We Built Instead
We automated the parts that passed. The assistant extracts the structured facts from the complaint, looks up the order and the last contact, and presents all of it with a suggested queue and its reasoning. The person still decides.
The measured effect was on time-to-decision rather than on headcount: the forty seconds became about fifteen, and the fifteen are spent on judgement rather than on looking things up in three systems.
Why the List Has Grown
We expected the refusal list to shrink as models improved. It has grown, because our scoping has got better rather than because the models got worse. Early on we automated things we should not have and learned it from incidents.
The additions are mostly cases where being wrong is silent. That category has turned out to matter more than difficulty, and it is not something a stronger model fixes.
What We Do Not Claim
We do not claim these tasks are permanently out of reach. Several would pass the first test tomorrow if the relevant context were recorded in a system, and sometimes the right project is that recording rather than the assistant.
We also do not present this as caution for its own sake. Declining work is a cost to us. We do it because a system that is wrong invisibly is worse than one that does less.