Letting an Assistant Query the Database
The demonstration is irresistible: ask a question in German, get a number from the ERP. The version we run looks nothing like the demonstration, and the difference is almost entirely about what the assistant is allowed to see.
The Query That Was Correct
A user asked how many orders were open for a customer. The generated SQL was valid, ran quickly and returned eleven. The correct answer was seven, because the table it counted includes cancelled orders and the business meaning of open excludes them.
Nothing failed. The query was syntactically correct, semantically reasonable and returned a number that looked exactly like an answer. This is the characteristic failure of generated SQL and it is why the demonstration is misleading: in a demonstration, someone who knows the schema is watching.
Why the Schema Is the Problem
A production ERP schema encodes decades of decisions: status flags whose meaning changed, tables kept for migration, columns whose names describe what they held in 2011. A model reading that schema infers meaning from names, which is exactly what the names do not reliably carry.
Shen and colleagues published MageSQL in 2025, work on improving text-to-SQL through in-context learning, and the framing that stayed with us is that the difficulty is concentrated in schema understanding rather than in producing syntax. Syntax is the part that was solved first and the part nobody was losing money on.
What We Expose
Not tables. A small set of views written by someone who knows the domain, each with a name and a documented definition: open orders, deliveries this month, overdue invoices. The view encodes the business meaning once, in SQL, reviewed by a person, and the assistant queries only those.
That turns an open-ended schema problem into a bounded one. The assistant can still combine views, filter and aggregate, and it cannot decide what open means, because that decision was made in the view definition by someone accountable for it.
| Allowed | Not allowed |
|---|---|
| Select from curated views | Any access to base tables |
| Filter, group, aggregate | Joins the view does not provide |
| A row limit, always applied | Unbounded result sets |
| Read-only credentials | Any write, ever |
What Every Answer Shows
The query, in full, below the number. Users do not read it and the people who matter do: when a controller disputes a figure, the conversation is about a visible SQL statement rather than about whether the assistant can be trusted.
We also show which views were used and the row count before aggregation. That last one has caught more errors than anything else, because a total over three rows and a total over three thousand look identical once they are a single number.
The Limits That Are Not Technical
A row limit on every query, applied by us rather than by the model, and a timeout. Both exist because the failure we most wanted to avoid is not a wrong answer, it is an assistant that puts a production database under load at month end because someone asked a question with a wide date range.
Read-only credentials, without exception, and on a replica where one exists. That is not a statement about the model's judgement. It is that a system where a wrong answer is embarrassing and a wrong write is an incident should not have the ability to produce the second.
How the Views Get Written
By a domain expert and a developer together, from the questions people actually ask. We collected two months of questions that had been sent to the controlling team, grouped them, and eleven views covered about eighty percent.
The remaining twenty percent are genuinely varied and go to a person, which is the same shape as every other system we build: the bounded majority is automated, the long tail is routed, and nobody pretends the tail is small enough to ignore.
What It Does Not Replace
The reporting system. We were asked twice whether this could replace scheduled reports, and it should not: a report that runs every Monday with a fixed definition is more reliable than a question asked slightly differently each week, and it is cheaper.
The assistant is for the questions between reports, the ones that would otherwise be an email to controlling. Measured that way it saved about six hours a week of someone else's time, which is a modest and real result rather than a transformation.
What We Do Not Claim
We do not claim generated SQL over a raw schema cannot work. With a clean, well-named, documented schema it works considerably better than our caution suggests, and we have seen it. Our customers do not have that schema, which is a statement about production systems rather than about the technique.
We also do not claim views eliminate wrong answers. A view can encode a definition the business has since changed, and then the assistant is confidently wrong in a way that is now our fault rather than the model's. Views are reviewed annually for exactly that reason.