What the Assistant Remembers, Anyone Can Write To
Memory across sessions is the most requested feature we have. It is also the only feature we have shipped that lets a user change what a different user is told, and that property was not obvious to anyone in the room when we agreed to build it.
Why Memory Is Requested
Because repeating yourself is annoying. A user who has said three times that they work on the older equipment variant reasonably expects the assistant to know that, and an assistant that remembers is noticeably better to use.
The implementation is easy: extract durable facts from a conversation, store them, and include the relevant ones next time. Two days of work for a version that demonstrates well, which is roughly where we were when someone asked who else those facts apply to.
The Property Nobody Names
Every other feature we ship is read-only from a user's perspective. They ask, the system answers from documents they are entitled to see, and nothing they do changes what anyone else is told. Memory breaks that, because a fact extracted from one conversation becomes an input to another.
In a shared or team-scoped memory, a user can state something confidently, have it stored as a durable fact, and change the answers other people receive. They do not need to intend it. Saying we always use the manual override is enough if the extraction step believes them.
The Research That Makes It Concrete
Dong and colleagues published work in early 2025 on memory injection against agents through query-only interaction, showing that an attacker who can only send ordinary queries can cause harmful content to be written into an agent's memory and influence its later behaviour.
Query-only is the part that matters. There is no privileged access, no compromised component and nothing that looks like an attack in a log. The mechanism is the feature working as designed, which is why a threat model that lists components will not find it.
| Memory scope | Who can affect whom |
|---|---|
| Per user | Nobody but themselves. Our default |
| Per team | Any member affects the team. Needs review |
| Per tenant | Any user affects everyone. We do not do this |
| Derived from documents, not conversation | Nobody. This is not really memory |
What We Ship
Per-user memory by default, which removes the cross-user property entirely and keeps most of the benefit, because the annoyance being solved is a user repeating themselves rather than a team sharing context.
Team-scoped memory exists for two customers who asked for it and understood the trade. It stores only facts a named person has confirmed, which turns memory from something the system infers into something someone asserts, with their name on it.
What Never Goes In
Anything that reads as a rule or an instruction. Preferences about format, equipment variant, language, role: those are stored. Statements like always approve orders under five hundred euros are not, regardless of who says them, because that is a policy and policies do not enter the system through a chat window.
The extraction step is constrained to a typed schema with a fixed set of fields for exactly this reason. A free-form memory that stores whatever seemed important is the version of this feature that produces the failure the research describes.
What the User Sees
Everything. There is a page listing what the assistant has stored about them, each item with the date and the conversation it came from, and a delete control on every row. That is partly a data protection requirement and mostly the only way anyone notices a wrong memory.
About one item in twenty gets deleted by users, and the deletions cluster in the first week after someone starts using the system, which suggests early conversations produce the least reliable inferences. We now weight recency in what gets stored.
What It Cost to Do Properly
The two-day feature took a fortnight. The extra time went into the schema, the visibility page, the deletion path and a rule that memory is never used to override a document: if a stored preference conflicts with the source, the source wins and the answer says which preference was set aside.
That last rule is the one we would keep if we could keep only one. It bounds the damage of a wrong memory to a wrong preference rather than a wrong fact, and it means the worst case is an answer in the wrong format rather than an answer that is untrue.
What We Do Not Claim
We do not claim per-user memory is safe from the user themselves. Someone can mislead their own assistant, which is a support problem rather than a security one, and the visibility page is the remedy.
We also do not claim our field list is complete. It covers the preferences we have seen matter, it will miss one, and the design assumption is that a missing field produces a slightly worse experience rather than an opening, which is the property we were actually buying.