Technical analyses
Since 2023, we have analyzed what works reliably in AI systems — using numbers, sources, and clearly stated technical limits.
The device assistant was used far less than expected, and the reason was that answering a question meant putting down a tool. Voice fixes that and adds a failure text never had: a misheard part number produces a confident answer about a different component. Pahwa et al. benchmark speech tool use. What we confirm before acting, and why the answer is spoken and shown.
Memory improved the assistant for individual users and quietly degraded the shared terminal, where one shift's preference shaped answers for the next. Al-Ratrout et al. name persona confusion in multi-user dialogue. Why identifying the speaker is the wrong first move, what a session boundary should be, and the preference that is safe to keep across everyone.
Two customers, essentially the same system, one renewed and one abandoned with everything working. The divergence was in the kick-off notes: one had written down what the assistant would not do, and the other had written down what it would. Vishwarupe et al. treat expectation management as a design concern. The four sentences we now insist on before any build starts.
A year of numbers on the agent that fixes failing tests. Tokens are the small half; review time on accepted and rejected proposals alike is the number that decides it, and rejected proposals still cost a review. Peng et al. compare cloud and on-premise inference economics for coding agents. What we measure, what changed when we measured it, and the threshold below which we would switch it off.
Reading back through everything published here, the same finding appears under a dozen headings: a constraint that lives in a prompt or a document is not a constraint. Besanson argues governance belongs in the architecture rather than attached to prompts and documentation. The eight places we learned it, the two where the architectural version was not available, and what the sentence does not cover.