State Belongs in a Store, Not in the Context
A long conversation is not a memory. Ours proved that by contradicting itself about a delivery date it had confirmed six minutes earlier, and the repair changed how we treat everything an agent decides.
The Contradiction
The assistant handled order enquiries. In one session it confirmed a delivery window, the customer asked two follow-up questions about packaging, and then it quoted a different window when asked to repeat it. Nothing external had changed. The earlier answer was still in the transcript.
Both answers were derived from context that contained the same facts. The second derivation simply weighted them differently, because by then the transcript had grown and the confirmation was no longer near either end of it.
Why Bigger Was Not the Fix
The obvious response is more window, and it does not work, because the problem is not capacity. A transcript is a record of what was said, not a record of what is true. Reading a decision back out of a conversation means re-deriving it, and re-derivation can differ.
Packer and colleagues framed this well by treating context as a memory hierarchy, with the model paging information in and out under its own control, in the manner of an operating system. The framing is what stayed with us: context is working memory, and working memory is not where durable facts live.
What We Changed
Every decision the assistant makes is written to a store the moment it is made: the confirmed window, the selected variant, the address chosen among several. Each entry has a value, a timestamp and the turn that produced it.
On every subsequent turn, that store is rendered into the prompt as a short block of established facts, separate from the transcript. The model no longer re-derives what it already decided; it is told.
| Lives in the transcript | Lives in the store |
|---|---|
| What was said, in order | What was decided, as a value |
| Tone and phrasing | Identifiers, dates, selections |
| Grows without bound | Bounded and typed |
| Re-derived on each read | Read back unchanged |
What We Deliberately Keep Out
The store holds decisions, not observations. Retrieved passages, tool outputs and intermediate reasoning stay out of it, because they are inputs to a decision rather than the decision itself, and promoting them turns the store into a second transcript.
We also do not let the model write to the store in free form. Entries go through a typed interface with a fixed set of keys. A model that can invent memory keys will, and a store nobody can enumerate is not inspectable.
What It Bought Us
The contradictions stopped, which was the point. The unplanned benefit was operational: when a session goes wrong, the store is a short readable record of what the assistant believed, which is far easier to reason about than a long transcript.
It also made handover possible. A session can be picked up by a human agent who reads twelve lines of state rather than eight minutes of dialogue.
What We Do Not Claim
We do not claim this is memory in any interesting sense. It is a key-value store with a rendering step, and it solves consistency rather than recall. Questions about what the customer implied three turns ago still go to the transcript.
We also do not claim the boundary is always obvious. Deciding whether something is a decision or an observation takes judgement, and we have moved things across that line twice after getting it wrong the first time.