The Answer That Lives in Two Documents
The question was whether a delivery was still inside the agreed window. The delivery date was in one document and the agreed window in another, and no single retrieved passage could ever contain both.
The Question That Needed Both
Users of a customer assistant asked, often, whether a particular delivery had been within the agreed period. That is one question to a person and two facts to a system: when the goods arrived, which is recorded on the order, and what the agreed window was, which is a clause in a framework contract.
Retrieval returned one or the other and never both, and the answers it produced were plausible and unreliable, because a model given only the delivery date will happily reason about a window it has assumed.
The Answer We Did Not Build
The technique everyone reaches for is a knowledge graph: extract entities and relations across the corpus, and traverse them at query time so that facts from different documents can meet. Gutiérrez and colleagues published work that month in this direction, building a graph over the corpus and using an association-based retrieval step so that facts spread across documents can be brought together in one hop.
It is the right general answer, and we did not do it. Before building a general mechanism we counted the queries: this pattern was about three percent of traffic, and every instance of it linked two documents through a contract number that both already carried as a field.
What We Built Instead
A join. The extraction step already writes structured records for both document types, both records already have the contract number, and answering the question is a query against those two tables followed by a date comparison in code.
The model is not involved in the comparison at all. It reads the question, calls a function with an order number, and receives a small structured result containing the delivery date, the agreed window and whether one falls inside the other. It then writes a sentence, which is the part it is good at.
| Situation | What we would build |
|---|---|
| Documents share an identifier | A join on the field. Deterministic and cheap |
| Link is implied by wording | Entity extraction, then a graph |
| The link is the answer itself | A person. This is a judgement, not a lookup |
| Under about five percent of queries | Measure before building anything general |
Why the Boring Version Wins Here
A join on an identifier is exact, auditable and free to run. There is no similarity threshold to tune, no extraction step to keep current, and when it returns nothing that is a fact about the data rather than a retrieval failure to investigate.
A graph earns its complexity when the link is not written down anywhere: when two documents refer to the same thing by different names, or when the relationship is implied by wording rather than recorded in a field. Our documents were not like that, and building a general mechanism for a link that already existed would have been elaborate machinery around a foreign key.
How We Found the Three Percent
We took a month of questions, sampled two hundred and classified them by hand into answerable from one document, answerable from two, and not answerable from documents at all. It took an afternoon and it decided the architecture.
That step is the one most often skipped, in our experience including by us. The temptation is to design for the hardest question anyone has asked rather than for the distribution, and the distribution is usually far more lopsided than it feels from inside a support conversation.
The Part That Was Genuinely Hard
Not the join. The hard part was that about one in nine orders referenced a contract number that had been superseded, and the correct window was in the successor. Answering with the original contract was wrong in a way no retrieval technique would have fixed, because the retrieval was working correctly on a stale link.
We handled it by following the succession chain in the query and returning both the applicable window and the contract it came from. Users see which contract the answer relies on, which turned out to matter more to them than the answer itself in disputed cases.
When We Would Change Our Minds
If the share of two-document questions climbed toward a fifth of traffic, or if a corpus arrived where the links are not identifiers, we would build the general mechanism. We keep the query classification running quarterly so that this is a number rather than an impression.
That is the actual position: not that graphs are unnecessary, but that a measurement should decide, and that the measurement is cheap enough that skipping it is never justified by urgency.
What We Do Not Claim
We do not claim a join replaces associative retrieval. It works precisely because our documents share a field, and a corpus where the connection is semantic rather than recorded needs the mechanism we chose not to build.
We also do not claim our three percent transfers. It is a property of one customer's questions and one document set, and the transferable part is the afternoon spent counting rather than the number it produced.