The Test Suite Is the Migration
Nobody wants to hear that the first phase of an AI-assisted modernisation is writing tests by hand. It is also the only version of the project we have seen work, and the reason is not about models at all.
What Was Actually Being Asked
A fifteen-year-old service, a language version out of support, and a proposal to have an assistant translate it. The code is not enormous and the translation itself is well within what current models do reliably.
The question nobody had asked is how anyone would know whether the translated system behaves the same. There was no test suite worth the name, and the behaviour of the old system was defined by the old system, which is a circular definition until someone writes it down.
Why the Tests Come First
Because a migration without them is not a migration, it is a rewrite with a hope attached. Every difference the translation introduces is invisible until it reaches a user, and the differences that models introduce are small and semantic rather than large and obvious: an integer division, a null handling convention, an ordering that was incidental.
Liu and colleagues published MigrationBench in 2025, a repository-level benchmark for code migration, and repository-level is the operative word. Translating a file is a different problem from migrating a system, and the second is dominated by whether behaviour is preserved across the whole.
What We Test First
Not the code. The boundaries: what goes in and what comes out at each external interface, captured from production traffic where we can and written by hand where we cannot. A characterisation test that records what the system currently does, including the behaviour nobody intended.
That last part is essential and uncomfortable. A fifteen-year-old system has bugs that downstream systems now depend on, and a migration that fixes them silently breaks things. We record current behaviour, mark the ones that look wrong, and get a decision on each rather than improving anything by accident.
| Phase | Who does it |
|---|---|
| Characterisation tests at the boundary | Us and their developers, by hand |
| Translation of leaf modules | Assistant, reviewed |
| Translation of core modules | Assistant, reviewed more carefully |
| Behaviour changes anyone wants | Separate project, after migration |
Leaves Before Trunks
We migrate modules with few dependents first, verify against the tests, and move inward. That is ordinary refactoring discipline and it matters more with an assistant than without, because a wrong translation deep in the dependency graph produces failures far from its cause.
It also produces a steady stream of verifiable progress, which matters commercially. A migration that runs for four months with nothing demonstrable is a migration that gets cancelled in month three regardless of how well it is going.
What We Refused
Letting the assistant improve anything while translating. Not the naming, not the structure, not an obviously silly loop. Every improvement is a behaviour risk during a phase whose entire purpose is that behaviour does not change, and improvements are also exactly what a model offers unprompted.
The instruction is explicit and the review checks for it. Two of the first twenty pull requests contained a tidy-up nobody asked for, both harmless, both rejected on principle, and after that the pattern stopped.
What the Six Weeks Bought
Two hundred and forty characterisation tests, of which eleven documented behaviour the customer considered a bug. The migration itself then took about seven weeks with an assistant doing most of the translation, and the tests caught nineteen behavioural differences, four of which we would not have found by reading.
The ratio is the point. Six weeks of unglamorous test writing made a seven-week migration possible and verifiable, and without it the same seven weeks would have produced something nobody could sign off.
Where This Does Not Apply
Systems with a good test suite already, where the first phase is short and the whole thing is much cheaper. And systems where the behaviour genuinely is not worth preserving, where the honest project is a rewrite with new requirements rather than a migration, and the assistant is helping with something else entirely.
Distinguishing those two is most of the value of the first conversation. A customer who says migrate this usually means one of them and has not decided which, and the answer changes both the cost and who needs to be involved.
What We Do Not Claim
We do not claim the assistant did the hard part. It did the volume, quickly and mostly correctly, and the hard part was deciding what correct meant and building something that could check it. That work is not automatable by the thing being automated.
We also do not claim our nineteen differences is a good rate or a bad one. It is one project, one language pair and one codebase, and the only claim we would make from it is that the number was not zero and we would not have known.