An Assistant for a Place With No Network
The technicians who most need the documentation work in a hall where there is no signal and no wireless coverage, for reasons involving reinforced concrete and a policy nobody was going to change for us.
The Constraint
No network in the hall, and a device that has to survive being dropped and to run a shift on one charge. That rules out calling a hosted model, and it rules out most of what we would normally build, because our architecture assumes a round trip to an index.
It also rules out the usual escape hatch of queuing questions until the device reconnects. A technician standing at a machine needs the torque figure now, and an answer that arrives when they are back in the office is not an answer.
What Fits
A small model, an embedding model and a subset of the corpus, running entirely on the device. Wang and colleagues published work in 2025 on an inference engine for deploying language models on mobile devices, which is the category of engineering that made this feasible at all in a form a technician would tolerate.
Feasible is doing a lot of work in that sentence. The model that fits is considerably weaker than what we run on servers, and the corpus that fits is a fraction of the whole, and both of those are product decisions rather than technical details.
The Decision Nobody Wanted to Make
Which documents go on the device. Everyone's first answer is all of them, which does not fit, and the second is the most important ones, which nobody could define. We resolved it by usage: the equipment in that hall, the procedures performed there, and the last three years of revisions.
That covered about nine percent of the corpus and about four fifths of the questions asked from that location, which we knew because the wired terminals at the hall entrance had been logging queries for a year. Without that log the scoping would have been an argument instead of a measurement.
| On the device | Not on the device |
|---|---|
| Equipment in that hall | The rest of the plant |
| Procedures performed there | Commercial and contractual documents |
| Last three years of revisions | The full revision history |
| A small model and index | Anything requiring the frontier model |
What It Says When It Cannot Answer
That the question is outside what it carries, and where the answer would be. Not a generic refusal: the device knows which document families it holds, so it can say this concerns the packaging line, which is not on this device, and offer to queue the question for when it reconnects.
That message is the difference between a tool people keep using and one they abandon. A device that fails silently on a fifth of questions teaches its users that it is unreliable; one that names its own boundary teaches them what it is for.
Quality, Honestly
On the questions it carries, the device answers correctly about eighty-one percent of the time against the server system's ninety-four on the same questions. That gap is real and we put it in the acceptance criteria rather than discovering it later.
The customer accepted it because the alternative was walking to the hall entrance, which takes four minutes each way. A worse answer immediately beat a better answer after eight minutes for exactly the questions the device was scoped to, and that framing is what made the trade explicit rather than embarrassing.
The Update Problem
The corpus on the device goes stale, and a stale specification is the failure mode we care most about in this domain. Devices sync when they are docked at shift end, and the assistant shows the age of its content on every answer.
If content is more than seven days old the answer carries a warning, and past fourteen the assistant declines specification values entirely while still answering procedural questions. Those thresholds came from the customer's own change frequency rather than from us.
What We Would Not Do Again
Start with the model. We spent the first fortnight on what would run and how fast, which was interesting and was not the hard part. The hard part was the scoping and the staleness policy, both of which are product decisions that a customer has to make and that we could have started on day one.
The technical work turned out to be the part with the least uncertainty, which is a pattern we have now seen enough times to expect and still have to remind ourselves about at the start of a project.
What We Do Not Claim
We do not claim a device assistant replaces the server one. It answers a defined subset in a place where the alternative is nothing, and every user also has access to the full system when they are somewhere with a network.
We also do not claim eighty-one percent is good. It is acceptable for this scope under this constraint, it would be unacceptable as a general system, and quoting it without the scope attached would be the kind of number that gets a project sold and then disappoints.