Technical analyses
Since 2023, we have analyzed what works reliably in AI systems — using numbers, sources, and clearly stated technical limits.
The fourth time we wrote an adapter so a model could query an order system, it took two days and looked exactly like the previous three. A standard tool interface removed the duplication and moved the remaining work somewhere it belongs: deciding what a tool should expose. What we standardised, what we deliberately did not, and the security property that a shared interface makes easier to state and easier to get wrong.
Eight customer systems, an afternoon each. None came out high risk, which was the boring part. The useful part was that writing down what each system does, what it decides and what a person still decides forced questions our design reviews had never asked, and two systems changed as a result. What we produce per system now, and why we say plainly that this is not legal advice.
Customers who ask about fine-tuning mean the model that writes the answer. On the two projects where training on customer data actually paid, it was the retriever: far cheaper, far smaller, and aimed at the failure that was actually costing us, which was finding the wrong passage rather than writing about the right one badly. What the training data has to be, and the maintenance nobody mentions.
We stripped names, addresses and customer numbers before sending tickets to a hosted model, and a colleague identified the customer from what was left in about ten seconds. Direct identifiers are the easy half. What we changed once we accepted that redaction bounds exposure rather than removing it, where we stopped sending data at all, and why a placeholder that keeps its meaning is worth more than one that does not.
Vector storage was a rounding error at two hundred thousand chunks and a line item at four million, because it grows with the corpus rather than with usage. Quantising the vectors cut memory by roughly three quarters at a retrieval quality cost we could measure and accept. What we measured first, the two-stage arrangement we ended up with, and the cheaper fix we should have tried before any of it.
Adding a third-party tool server takes about a minute, which is the value of a standard interface and the reason nobody reviews what they add. A tool description is text the model reads and trusts, so it is an input channel as much as a capability. What we check before one runs, what we refuse outright, and why the credentials matter more than the code.
The passage was correct, well written and three years out of date. Our index held four revisions of the same manual with no notion of which applies, because similarity search has no opinion about time. What we changed in the metadata, why the default has to be the current revision rather than the best match, and the question type we now refuse to answer without a date.
A customer asked whether their agent could talk directly to their supplier's agent. Technically yes; the hard part is that neither side can see why the other decided what it did, so a wrong order has no diagnosable cause. What we build instead of a free-form exchange, the three things we insist a cross-boundary agent interaction records, and where we said no outright.
A procurement department asked for the energy footprint of an assistant for a sustainability report, and we had an invoice and no way to answer. What we can measure honestly, what we can only estimate with a stated range, and the design decision the exercise changed: a background summarisation step nobody was waiting for that ran on every document and could run on a fraction of them.
Accuracy improved for a year and escalations from one user group did not move, because the assistant answered in the language of a manual written for engineers. Simplification is not a prompt instruction that says be simple; done carelessly it drops the qualifier that made the sentence true. What we simplify, what we refuse to, and how we measured whether it worked.
Memory across sessions is the most requested feature we have, and it is the only one that lets a user change what a different user is told. Dong et al. show memory can be poisoned through ordinary queries alone, with no privileged access. What we scope, what we never write to memory, and the review that turned a two-day feature into a two-week one.
A customer asked why their assistant could not just look it up when the manual is silent. Web access answers more questions and makes every answer unattributable to a source they control, which in a regulated industry matters more than the answer. Where we do allow it, the two properties a source needs before it goes in the corpus, and what we do instead when the manual is genuinely silent.