Home AI Solutions Ready-made Solutions Peers & Simulation RAG & Retrieval Use Cases Frameworks Blog Deutsch Contact Us
Back to the blog

Why Our Assistants Do Not Browse the Open Web

A customer asked why their assistant could not just look it up when the manual was silent. It is a reasonable question with a specific answer: because then nobody can say afterwards where the answer came from, and in their industry that matters more than the answer.

The Question, Fairly Put

Their technicians ask things the manual does not cover: a fault code from a third-party component, a material compatibility question, a regulation reference. The assistant says it cannot help, and a search engine would have answered in seconds.

That is a real gap and we do not dismiss it. The reason we do not close it with web access is not that the answers would be bad. It is that an answer from the open web has a property their process cannot accommodate.

Documentschunks · vectors Indexvector + keywordgraph Query Hybrid Searchrrf Rerankercross-encoder Answerwith sources
Documents are chunked, embedded and indexed — vectors plus keywords. 1/4

The Property That Rules It Out

Their maintenance records have to say which document a decision was based on, and that document has to be one they can produce years later in the state it was in. A web page is not that: it changes, it disappears, and it was never under their control.

So an answer from the web is unattributable in the sense that matters to them, even when it is correct and even when we store the URL. The retained artefact would be our copy of someone else's page, which is a different thing from a controlled document.

The Research on Agents That Do Search

Zheng and colleagues published DeepResearcher in April 2025, training agents to carry out research in real web environments rather than in a simulated retrieval setting, which is a serious treatment of the capability we are declining to use here.

We cite it because our position is not that this does not work. It is a capability we expect to be useful and are watching, and the reason it stays out of these deployments is a property of the customer's process rather than of the technique.

SourceAllowed in the corpus
Customer documentsYes. Versioned and controlled
Supplier documentation, licensedYes, with a retained copy
Standards bodies, subscribedYes, same terms
Open web at query timeNo. Cannot be retained or fixed

What Goes In Instead

Two properties decide it. We must be permitted to retain a copy, and the copy must be identifiable by version. Everything meeting both goes into the corpus through the normal ingestion path, including things people assume are off limits: supplier documentation is usually licensable, and standards bodies sell subscriptions that permit exactly this.

That process closed most of the gap. The third-party fault codes that started this were in a document the supplier was happy to license for a modest fee, and nobody had asked because everyone assumed the answer would be no.

Where We Do Allow It

Internal tooling used by our own engineers, where the output is read by someone who knows to check it and no record is created. And one customer-facing case: a competitive-intelligence assistant where the whole point is public information and nothing downstream depends on provenance.

The distinction is not the risk of a wrong answer. It is whether anything is retained that someone will later have to justify. Where nothing is retained, web access is a convenience with ordinary risks.

What the Assistant Says When It Cannot Help

It names what it searched and what it did not, which sounds pedantic and is the difference between a dead end and a next step. Not found in the maintenance documentation for this equipment; this component is documented by the supplier and is not in your corpus.

That message caused four documents to be licensed in the first six months, because it told someone with a budget exactly what was missing. An assistant that says it cannot help teaches users to stop asking; one that says what is absent turns a failure into a request.

The Version We Would Build

If a customer needed it, a separate mode with its own visual treatment: answers marked as coming from public sources, excluded from anything that gets recorded, and never mixed with corpus answers in the same response. Mixing is the specific thing we would refuse, because a paragraph combining a controlled document and a web page inherits the weaker property.

We have not built it because nobody has asked for it with that constraint attached, and building a mode nobody has scoped is how features arrive without a decision behind them.

What We Do Not Claim

We do not claim web-searching assistants are worse. For many purposes they are obviously better, and a research assistant without web access would be pointless. Our position is scoped to systems whose answers enter a record someone is accountable for.

We also do not claim our corpus is complete. It is bounded by what can be licensed and retained, some things cannot, and the honest consequence is an assistant that declines more often than one with the whole web behind it.

Sources