What the System Costs to Run, in Energy Rather Than Euros
The question arrived from a procurement department rather than from engineering, which is how most useful questions arrive. What is the energy consumption attributable to this system, and we had a monthly invoice and no way to convert it into anything a sustainability report can use.
The Question We Could Not Answer
Their sustainability reporting needed a figure for the systems they operate, and ours was one of them. We had cost per month, tokens per month and requests per month, none of which converts to energy without knowing hardware, utilisation and data centre efficiency that a hosted provider does not publish per call.
For the self-hosted parts we were better placed, because the hardware is ours and its consumption is measurable. That split, between what we can measure and what we can only bound, turned out to be the honest structure for the whole answer.
What the Research Gives Us
Rehman and colleagues published work in early 2025 analysing energy consumption in the fine-tuning of language models for summarisation, measuring rather than estimating what different configurations consume.
Studies of this kind are what make a defensible estimate possible at all. We cannot measure a hosted provider's per-request energy, and published measurements on comparable hardware and model sizes give a range that we can state as a range rather than presenting a single number with false precision.
| Component | What we report |
|---|---|
| Self-hosted models on our hardware | Measured, from power draw |
| Vector index and application servers | Measured, same way |
| Hosted model calls | Estimated range, method stated |
| Model training | Not attributable. We do not train |
How We Split It
Measured where we own the hardware. Our self-hosted inference, the index and the application servers run on machines whose consumption we can read, apportioned by utilisation, and that part of the figure carries no estimation at all.
Bounded where we do not. For hosted calls we give a range derived from published measurements for comparable model sizes, with the assumptions written next to the number. A customer's auditor can disagree with our assumptions, which is exactly what we want: a number they cannot interrogate is worse than a range they can.
The Decision It Changed
The same background summarisation step that had turned out to be thirty-eight percent of our model spend was also the largest energy component, and for the same reason: it ran on every ingested document whether or not anyone would ever ask about that document.
We already knew from measuring retrieval that a large majority of the corpus is never retrieved. So the step now runs on demand for documents that are actually retrieved, with the summary cached afterwards. Same behaviour for users, a fraction of the work, and the reduction shows up in both the invoice and the energy figure.
The Thing That Did Not Move
Latency, and therefore user experience, because the summarisation was never on the path of anything a person waited for. That is what made the change easy, and it is also a warning: the steps that are easiest to make efficient are the ones nobody notices, which is precisely why they grow unchecked.
We now review background work quarterly with one question. Does this run because someone needs the result, or because it was cheaper to run it on everything than to decide what needs it? The second answer was true twice.
What We Put in the Report
A figure with its method, a range where a range is honest, and an explicit statement of what is excluded: model training, which we do not do, and the provider's own infrastructure overheads, which we cannot see.
Writing down what is excluded was the part the customer's reporting function valued most. A number without a boundary cannot be audited, and they had been given several of those by other suppliers.
Whether It Is Worth Doing Unasked
For most customers, not yet. The exercise took about a week and only two of our customers have needed it, both in sectors with reporting obligations. We do not push it on anyone else, because a measurement nobody uses is overhead dressed as diligence.
What we do keep is the instrumentation, which is a byproduct of the cost tracking we already run through the gateway. Producing the figure again for the next customer who asks is now a day rather than a week, and that seems like the right level of investment.
What We Do Not Claim
We do not claim our estimate for hosted calls is accurate. It is a range built on assumptions we state, and the true figure could sit outside it if the provider's hardware or efficiency differs materially from the published measurements we relied on.
We also do not claim the reduction we made was primarily environmental. It was a cost optimisation that had an energy benefit, and describing it the other way round would be the kind of framing this subject attracts and does not need.