The Call That Was Shaped Right and Meant Nothing
The tool call passed every check we had. The JSON parsed, the field names were right, the types matched, and the order number in it referred to nothing at all. Our validation had been answering a question nobody was asking.
What Passed and What Broke
A customer asked to change the delivery address on an order. The step that turns a request into an operation produced a call to our update function with a well-formed payload, and the target system rejected it because the order identifier did not exist.
Every check between the model and the system had passed. That is the uncomfortable part: we had built validation that confirms an argument looks like an order number without confirming that it is one, and those are entirely different guarantees.
Two Questions That Look Like One
Shape and reference. Shape asks whether the value is a string of the right pattern and length. Reference asks whether the thing it names exists and is the thing the user meant. A schema answers the first and is structurally incapable of answering the second.
Once we said that out loud, the fix was obvious and the omission was embarrassing. The model had been asked to produce an identifier from a conversation, which is a lookup, and we had accepted its answer without performing the lookup ourselves.
The Research on Making Calls Faithful
Wang and colleagues published work that summer on generating API calls faithfully and efficiently, addressing the gap between output that satisfies a specification and output that correctly reflects the intent behind the request.
What we took from it is the separation rather than a specific technique: constraining the form of a call and ensuring the call means the right thing are different problems, and a system that only invests in the first will produce well-formed nonsense with high reliability.
| Check | What it can tell you |
|---|---|
| JSON schema | The payload has the right shape |
| Enumeration of allowed values | The action is one we support |
| Lookup against the system of record | The identifier exists |
| Confirmation with the user | It is the one they meant |
What We Do Now
The model does not produce identifiers. It produces a description of what the user referred to, and a deterministic step resolves that description against the order system. If the resolution returns nothing, or more than one candidate, the assistant asks rather than guesses.
That last case turned out to be common. About one request in twelve refers to an order ambiguously, and previously the model resolved the ambiguity silently and confidently. Now the user sees two orders and picks, which is both correct and faster than a wrong update discovered later.
Why We Stopped Blind Retries
Our first patch was to feed the rejection back and let the model try again. It works often enough to be tempting and it has a specific failure: given a rejection, a model will produce a different plausible identifier rather than concluding that it does not know one.
We now retry only when the failure carries information the model can act on, such as a missing required field. A rejection meaning no such record ends the attempt and routes to resolution or to a person, because there is nothing to correct by trying harder.
The Argument No Schema Catches
The remaining risk is an argument that exists, is well-formed, and is wrong. A valid order number belonging to a different customer passes shape, passes lookup and is still the wrong call, and there is no purely local check that catches it.
For that we use scope rather than validation. The credentials the step operates under can only see and modify the requesting customer's records, so an identifier outside that scope fails at the system boundary rather than in our logic. Least privilege doing work that validation cannot.
What It Cost
One extra lookup per operation, which is a database read against a system we already talk to, and one additional turn of conversation on the ambiguous cases. Latency rose by a few tens of milliseconds and the wrong-target rate on our labelled set went to zero.
The larger cost was conceptual. It required accepting that the model is not the component that decides which record is meant, which sounds like a reduction in capability and is really a correction to where the capability belonged in the first place.
What We Do Not Claim
We do not claim resolution removes the class. A user can describe the wrong order confidently and our system will resolve it faithfully, which is a correct outcome from an incorrect premise, and only a confirmation step catches it.
We also do not claim schemas are the wrong tool. They are necessary and they do their job. The claim is narrower: they answer a question about form, and a system that treats a passing schema as a passing call has not checked the part that fails.