Conversation Is Not Control Flow
Two agents talking to each other solved our task in the demo and became undebuggable in production. The pattern was not wrong — our use of it was. Here is where we draw the line now.
The Demo That Did Not Survive
The task was document intake: classify an incoming document, extract fields, check them against a register, and either file or escalate. We built it as two agents in conversation — one proposing, one reviewing — because that read cleanly and handled edge cases without us enumerating them.
It worked. Then it stopped working for about one document in thirty, and we could not say why. There was no step to point at. The transcript showed two agents agreeing on something wrong, at length, in fluent prose.
What the Pattern Is Good At
AutoGen, published by Wu and colleagues that August, is the clearest statement of this approach: applications built from multiple agents that converse to accomplish a task, with conversation patterns programmable in natural language or code, and humans and tools mixed into the exchange.
The strength is real. Conversation absorbs cases you did not plan for, because the agents can ask each other rather than fail on an unexpected branch. For exploration and for problems whose shape you do not know yet, that is exactly right.
Where It Cost Us
Production asks different questions. Which step failed. What was the state when it failed. Will a retry be safe. A conversation answers none of these directly: the state is spread across a transcript, the step boundary is implicit, and a retry replays a dialogue rather than resuming a process.
Our one-in-thirty failure was two agents converging on a misread field and reinforcing each other. In a state machine that is a validation failure at a named step. In a conversation it is a paragraph.
What We Do Now
The process is an explicit state machine: named steps, a typed state object, and a defined transition out of each step. Agents live inside the steps, where a model is genuinely better than a rule — classifying an ambiguous document, deciding whether two addresses are the same entity.
The transition itself is code. It checks the extracted fields against a schema, and a step either completes, fails or escalates. Retry resumes from the last completed step rather than replaying a discussion.
| Question production asks | How each answers it |
|---|---|
| Which step failed | Conversation: read the transcript. State machine: it is in the record |
| What was the state | Conversation: reconstruct it. State machine: one typed object |
| Is a retry safe | Conversation: unclear. State machine: resume from last completion |
| Where does a new check go | Conversation: into a prompt. State machine: at the transition |
Where We Still Let Them Talk
We have not banned the pattern. Inside a single step, where the output is checked before it leaves, a proposer and a reviewer produce better results than one call — particularly for judgement calls where a second pass catches overconfidence.
The rule we hold is about scope: agents may converse within a step, never across one. The boundary between steps is where state is written down, and state written down is what makes a system operable.
What We Do Not Conclude
We do not conclude that conversational frameworks are the wrong tool. For research, for prototypes, and for problems where the sequence genuinely is not known in advance, the flexibility is the point and our structure would only get in the way.
Nor do we claim the state machine makes the agent correct. It makes the agent's mistakes locatable, which is a different and more modest claim — but it is the one that decides whether a system can be run by people who did not build it.