The Right Answer From the Wrong Revision
The passage was correct, well written and three years out of date. Similarity search had done exactly what it does, and nothing in the pipeline had been asked to consider that the same sentence exists in four revisions with different numbers in it.
Four Copies of the Same Sentence
The customer keeps every revision of every manual, which is correct practice and required for their industry. We had indexed all of them, which seemed obviously right at the time: more material, better coverage.
The result is that a query about a torque specification retrieves four nearly identical passages differing in one number, and the reranker chooses between them on the basis of textual similarity to the question, which has nothing to do with which one is in force.
Why Similarity Has No Opinion About Time
An embedding places a passage according to what it says. Two revisions of a paragraph say almost the same thing, so they sit almost in the same place, and no amount of better embedding separates them because they are not semantically different in the way the model was trained to notice.
The information that distinguishes them is not in the text at all. It is in the document's revision number and effective date, which is metadata, and we had been treating metadata as something to display rather than something to retrieve on.
The Research on Time-Sensitive Retrieval
Abdallah and colleagues published TempRetriever in early 2025, a retrieval approach for time-sensitive questions that brings the temporal dimension into the retrieval step rather than leaving it to be sorted out afterwards.
The framing we took is that a question can carry a time even when it does not state one. Someone asking what the tightening torque is means now, and a system that treats that question as timeless will answer it from whichever revision happens to match best.
| Question | What it means |
|---|---|
| What is the torque specification? | As of today. Current revision only |
| What was it before the change? | A specific earlier revision |
| When did it change? | A comparison, not a retrieval |
| What applies to machine number 4711? | The revision current at its build date |
What We Changed
Every chunk carries the revision and its validity period, and retrieval filters to the revision in force by default. Superseded revisions are still indexed and are reachable only when the question or the interface asks for them, which turned a ranking problem into a filter.
The last row of the table is the one that made this genuinely useful rather than merely correct. For equipment, the applicable revision is the one current when that unit was built, and once the metadata existed we could answer that question at all, which we previously could not.
The Default That Matters
Current revision, always, unless something says otherwise. It is tempting to let the ranker decide and show the user which revision it chose, and that is worse: a user who is shown four options with dates will pick one, and the one they pick under time pressure is the first.
Where the question genuinely is historical, our interface has to be asked. That is friction and it is the right place for friction, because a wrong answer about a current specification is expensive and a slightly slower path to an old one is not.
What We Refuse Without a Date
Questions about a document set where the revisions disagree and the query gives no way to choose. If a user asks what the procedure is and their equipment spans three revisions, we ask which unit rather than answering for one of them.
That refusal was unpopular for about a fortnight. It became popular when the first person noticed that the previous behaviour had been answering confidently for whichever revision happened to be indexed most recently.
What It Cost
Extracting revision metadata reliably, which was three weeks of unglamorous work because the revision is in a different place on documents from different decades. That is the whole cost, and it is entirely in the ingestion pipeline rather than in retrieval.
Retrieval itself got slightly faster, because the filter removes three quarters of the candidate set before ranking. That was not the goal and it is a reasonable illustration that indexing everything indiscriminately was not free either.
What We Do Not Claim
We do not claim this generalises to corpora without explicit revisions. A knowledge base where articles are edited in place has the same problem with none of the metadata, and the honest answer there is that the retrieval layer cannot fix what the source system does not record.
We also do not claim our filter handles partial supersession. A revision that changes one section leaves the rest in force, and we treat the whole document as superseded, which is conservative and occasionally hides a passage that was never revised.