We Classified Our Own Systems Before Anyone Asked
European rules on artificial intelligence began applying in stages from 2024 onward, and the first obligations took effect in early 2025. We went through our systems before a customer asked us to, and the exercise was more useful than the result.
Why We Did It Early
Not out of virtue. A customer in a regulated sector was going to ask eventually, and answering that question under time pressure with an auditor in the room is a much worse experience than answering it on a quiet afternoon with the people who built the thing.
We also expected the classification to be uninteresting, and it was. Our systems extract fields from documents, answer questions from documentation and route requests, none of which sits in the categories the regulation treats as high risk. That was the result within an hour for six of the eight.
The Two That Took Longer
One system produces a recommendation that feeds into a decision about a supplier relationship, and one summarises information that a person uses when handling a complaint. Neither is a decision about a person in the sense the rules are concerned with, and both required us to write down precisely why not.
Writing that down is the work. It forced us to state, in a sentence a non-engineer can check, what the system outputs, who acts on it and what they are expected to do with it. In one case the honest sentence was uncomfortable enough that we changed the system rather than the sentence.
What the Research Adds
Sillberg and colleagues published an assessment of the European framework arguing that it is a reasonable starting point while falling short in specific respects, which is a useful corrective to reading the text as though every question has a determinate answer.
That matters practically. Several of our judgements sit in areas where reasonable people would disagree, and a paper documenting that the framework has gaps is a better guide to how confident to be than a checklist that implies certainty.
| What we write per system | Why |
|---|---|
| What it outputs, in one sentence | The basis for every other answer |
| Who acts on the output and how | Decides whether a person is in the loop |
| What data goes in, and where from | Data protection and provenance |
| What happens when it is wrong | The question nobody had asked |
The Question That Changed Things
What happens when it is wrong. Six of eight had a clear answer. Two did not, and in both cases the reason was the same: the output flowed into a process where nobody was designated to notice an error, so being wrong had no consequence until it had a large one.
That is not a regulatory finding, it is an engineering finding, and we would not have made it without the exercise. Both systems now have a named recipient for their output and a defined route for disputing it, which took a week and is the most concrete thing that came out of the whole review.
The Transparency Part
Users are told they are interacting with an automated system, which we had been doing already, and the wording got clearer. Our assistants now say what they can and cannot do in the first message rather than in a page nobody opens.
That change came from reading obligations rather than from a usability study, and it improved things measurably: escalations arrived with more context, because a user who knows the assistant cannot approve a credit note asks for a person earlier rather than after four failed attempts.
What We Give Customers
A short document per system, three pages, in German and English, covering the four rows in the table plus the models used and where they run. It is written to be read by their compliance function rather than by us, which mostly meant removing our vocabulary.
We update it when the system changes materially, and materially means the output, the recipients, the data sources or the model. That list is short on purpose, because a document that has to be revised for every deployment will not be revised at all.
What It Cost and What It Is Worth
About two weeks across eight systems including the writing, plus the week of engineering the review triggered. Two customers have since asked for exactly this document, and in both cases producing it immediately was worth considerably more than the fortnight.
The unexpected benefit was internal. New engineers read the system document before the code, and it answers questions the code cannot: who this is for, what they do with it, and what the failure looks like from their side.
What We Do Not Claim
This is not legal advice and we are not qualified to give it. We are describing what we did to prepare, our customers have their own counsel, and where our reading and their counsel differ, their counsel decides.
We also do not claim a classification is durable. Ours reflects the systems as they are and the guidance as we understood it at the time, and both change, which is why the review is scheduled annually rather than treated as finished.