We Do Not Sell a Hallucination Detector
Procurement documents ask for it by name, usually with a target percentage attached. We do not offer one, and explaining why has become a standard part of our first meeting rather than an awkward moment in the third.
What Is Being Asked For
A component that reads an answer and says whether it is true. Usually with a number: detection rate above ninety-five percent, false positives below five. Written by someone sensible who is trying to specify a quality requirement in a form a contract can hold.
The requirement is legitimate. What is being asked for as a component does not exist in the form the name implies, and taking the money to build one would be selling a name rather than a capability.
What Similarity Can and Cannot Certify
Sinha published work in late 2025 deriving limits on what embedding-based checks can establish about whether a generated answer is supported by its retrieved sources, arguing that similarity in embedding space cannot certify the property that the name detection implies.
Our own measurements agree from the other direction. A similarity check between answer and source catches an answer that has drifted from the passage, which is a real failure and the easy one. It passes an answer that quotes accurately and draws a conclusion the passage does not support, which is the failure that reaches customers.
The Three Checks We Actually Build
First, is there a source at all: retrieval returned nothing above threshold, so no answer is generated. That is deterministic and it removed the largest share of our unsupported answers.
Second, does the quoted text exist in the retrieved material, as a string comparison rather than a similarity. Third, do the extracted values reconcile with each other and with the systems of record. Each catches a specific, nameable failure, and none of them is a detector.
| What people ask for | What we provide |
|---|---|
| A truth check on the answer | A refusal when nothing was retrieved |
| A confidence score | A quotation that must exist verbatim |
| A hallucination rate below a target | A measured error rate on a labelled set |
| Detection of any wrong statement | Cross-field and system-of-record checks |
What We Put in the Contract Instead
An error rate on a named labelled set, measured monthly, with the set itself as an appendix. That is a commitment we can hold and a customer can verify, and it is about the system's output rather than about a component's capability.
It also survives contact with an auditor better than a detection rate. A percentage attached to a detector invites the question of how it was measured, and the honest answer usually reveals that the detector was evaluated on the failures it was designed to catch.
The Uncomfortable Part of the Conversation
That the remaining errors are not detectable by anything we can build, and the answer to them is process rather than technology: a person checks, or the system declines, or the output feeds a step where being wrong is cheap.
Customers accept this more readily than we expected. What they object to is discovering it after signing, and a supplier who says it in the first meeting is easier to trust on the things they do claim.
Where a Model Checking a Model Does Help
As a filter before a human, not as a gate. We use a second model to flag answers that look unsupported, and flagged answers go to review rather than being blocked. It raises the density of real errors in the review queue, which is worth something.
It is not a detector because it is not trusted to be right. A flag that routes work to a person has a different failure profile from a flag that suppresses output, and conflating the two is how a useful heuristic becomes a false guarantee.
How We Answer the Tender Question
By restating the requirement in terms we can meet, in writing, in the response: we do not offer hallucination detection, we offer a measured error rate on your data with a named test set, a refusal path when evidence is absent, and verification against your systems.
We have lost one tender on that answer and won four where the buyer said it was the only response that did not promise the impossible. That is a small sample and it is the reason we have kept doing it.
What We Do Not Claim
We do not claim automated checking is worthless. Our quotation check is automated, our reconciliation is automated, and the flagging model earns its cost. The claim is about what a component can certify, not about whether automation helps.
We also do not claim the research settles the question. It derives limits under stated assumptions, other approaches may do better than embedding similarity, and our position rests mainly on our own measurements, which are narrower and closer to the systems we run.