The Filter That Blocked the Work
A technician asked what happens if two cleaning agents are mixed. The filter classified it as a request for hazardous information and refused, which is defensible in general and precisely wrong in a plant where that question is a safety procedure.
The Refusal
The plant runs a maintenance assistant over its own documentation, which includes safety data sheets. A technician asked what happens if a particular acid cleaner meets a particular chlorine-based one, which is a question with a documented answer and an obvious operational reason for asking it.
The content filter classified it as seeking information about producing a hazardous substance and declined. The answer was in a document the technician is required to have read, sitting in the index, and the assistant would not repeat it.
Why This Is Not a Tuning Bug
Kumar and colleagues published work in 2025 on guardrails showing that reducing one kind of error increases the other, that there is no configuration that avoids both, and that the choice is a trade rather than a matter of getting the settings right.
That framing is what stopped us treating this as a defect to be fixed once. Every threshold we set is a decision about which mistake we prefer, and a filter tuned to never block legitimate work will let something through that we would rather it had not.
What We Measure
Both sides, deliberately, because measuring only refusals produces a system that never refuses anything. We track blocked requests that a reviewer judges legitimate, and we run a set of adversarial prompts weekly that the filter should catch.
The first number was seven percent of refusals when we started, which is high enough that users learn to stop asking. The second is the one nobody wants to look at, because a filter that has never blocked something genuinely bad has not been tested, it has just not been attacked.
| Setting | What you get |
|---|---|
| Strict everywhere | Blocks legitimate work. Users stop asking |
| Loose everywhere | Passes things that need judgement |
| Per corpus and per user role | More configuration, better fit |
| Refuse plus route to a person | Slower, and nothing is silently lost |
What We Changed
The filter is configured per deployment rather than globally. In the plant assistant, questions about chemical interaction are expected traffic and the threshold reflects that; in a customer-facing assistant on the same platform, they are not, and it does not.
That sounds obvious and it required admitting that a single safety configuration across deployments was a convenience for us rather than a property anyone wanted. The plant's safety officer signed off the configuration, which is where that decision belongs.
Refusals Are Defects With Tickets
Every refusal is logged with the question, and a sample is reviewed weekly by someone who can say whether it should have been refused. The ones that should not have been become tickets, and the ticket is against the filter configuration rather than against the user.
That process changed the number from seven percent to under two over a quarter. More importantly it changed the culture around the filter: it is a component with a defect rate rather than a moral position, and people report it when it is wrong.
Where We Accept Over-Blocking
Anything touching personal data of employees, and anything that would produce a legal opinion. In both cases we would rather refuse a legitimate question than answer one we should not, because the cost asymmetry is severe and the refusal has a clear next step: ask a specific person.
The refusal message names that person's role. A filter that says it cannot help teaches users that the system is unreliable; one that says this goes to the works council or to legal is doing the routing that the organisation wanted anyway.
The Part We Got Wrong Twice
Filtering the output as well as the input, with the same configuration. An answer quoting a safety data sheet contains exactly the vocabulary that an input filter is looking for, so our output filter blocked correct answers to questions the input filter had allowed.
Input and output need different rules, and the output rule is mostly about whether the content is supported by a retrieved document rather than about what words appear in it. Once we separated them the problem disappeared, and it had cost us a fortnight of confusing reports.
What We Do Not Claim
We do not claim our configuration is right for anyone else. It is right for one plant with one safety officer who accepted it, and the same settings in a consumer product would be indefensible.
We also do not claim two percent is a good false-refusal rate. It is where the measurable complaints stopped, which is not the same thing, and the remaining refusals include some we have simply not heard about because the user gave up rather than reporting it.