German Questions, English Documents
A German question about a Wartungsintervall does not retrieve an English passage about a maintenance interval unless something in the pipeline knows they are the same thing. For a long time nothing in ours did, and the failure looked like poor coverage rather than a language problem.
A Failure That Looked Like Something Else
Our support metrics showed German-language conversations escalating to a person about twice as often as English ones. The first explanations we reached for were about phrasing, or about German users asking harder questions, neither of which survived contact with the logs.
The actual cause was mechanical. The documentation is written in English, the queries arrive in German, and the retrieval step was matching text against text in a way that has no reason to connect the two vocabularies.
The Four Options
Translate the query into English before searching. Translate the documents into German and index both languages. Index the English text but with German metadata and headings. Or use an embedding model that places both languages in one shared space.
We built all four against the same labelled set, which for this purpose had to be rebuilt: sixty German questions with the correct English passage marked by someone fluent in both. That set took longer to produce than three of the four implementations.
What the Measurements Said
Translating the query worked well and came third. It is simple, it is cheap, and it fails on exactly the terms that matter most: product names, compound nouns that are half English already, and the domain vocabulary where a translation model picks the general word rather than the technical one.
Translating the documents scored highest on retrieval quality and we did not choose it, because it doubles the corpus, doubles the re-indexing cost, and creates a maintenance obligation where a German passage can silently fall behind its English original.
| Approach | Result on our set |
|---|---|
| Translate the query | Good, fails on domain terms |
| Translate the documents | Best retrieval, doubles maintenance |
| Multilingual embeddings | Close second, no extra corpus |
| German metadata only | Helped headings, not body text |
What We Chose and Why
A multilingual embedding model, scoring close behind document translation with none of its ongoing cost. Sturua and colleagues published jina-embeddings-v3 that September, a multilingual model with task-specific adapters, which is representative of what became practical around then.
The deciding factor was not the score. It was that this option adds nothing to maintain: one corpus, one index, and the language handling sits inside a component we were already going to have. Every other option added an artefact that can drift.
The Part That Still Needs Handling
Compound nouns. German technical writing produces terms that appear nowhere in the English text as a unit, and where a user searches for a Wartungsintervallüberschreitung the useful match is spread across an English sentence rather than concentrated in a phrase.
Our hybrid index carries most of this, because the keyword side matches part numbers and model designations that are identical in both languages, and those are frequently what a compound noun is anchored on. It is a partial answer and we have not found a complete one.
What We Tell Customers About Answer Language
The answer comes back in the language of the question, and it cites an English source. Some customers dislike that and ask for the source to be translated too, which we advise against: a translated quotation is our translation of their document, presented as their document.
Instead the citation shows the original passage with a translation beside it, marked as ours. That distinction has mattered twice in disputes about what a manual actually said, which is a small number and both times it was the whole point.
Why This Is Not Only About German
The same measurement applies to any pairing where the question language and the document language differ, and the point generalises past language: a retrieval failure that concentrates in one segment of users will usually look like a quality problem before anyone thinks to segment the metric.
We now break down retrieval quality by query language as a matter of course. It cost nothing to add and it is the reason we found this at all, since the overall numbers looked acceptable throughout the period when half our users were being served badly.
What We Do Not Claim
We do not claim multilingual embeddings are the right answer generally. Document translation scored higher, and for a small stable corpus where the maintenance burden is manageable it is a defensible choice we would make.
We also do not claim our sixty-question set is representative of German technical queries. It is one domain, written by one bilingual expert, and its main virtue is that it exists at all where previously we were reasoning about the problem without measuring it.