Designing Migration Instructions: Modernising Without Changing Behaviour

A request to modernise implies improvement, and improvement implies change. In a migration, change is the one thing you are trying to avoid.

Migration is where agents look best and behave worst

Framework migrations appear ideal for automation. The transformation is largely mechanical, the target is documented, and the work is tedious enough that nobody wants it. Teams reach for an agent, and the first hundred files go beautifully.

Then the problems arrive, and they are not the ones anyone anticipated. The agent modernises code that was deliberately written the old way. It changes behaviour while changing form. It applies the new pattern inconsistently across files, because it learned the pattern gradually. And none of this is visible in a diff that is thousands of lines long and, line by line, entirely reasonable.

Separate the mechanical from the semantic

The first structural decision: most migrations contain two different kinds of work, and mixing them is what makes review impossible.

That last row is worth sitting with. If a transformation is genuinely mechanical, an AST-based codemod does it deterministically, across ten thousand files, in seconds, with no possibility of creative interpretation. An agent is the wrong tool — slower, more expensive, and capable of being inventive.

Use the agent for the residue: the cases the codemod could not handle, and the transformations that require understanding what the code means.

The instruction that matters most

Behaviour preservation has to be stated explicitly, because the request to "modernise" implies improvement and improvement implies change.

## Scope of this migration
Convert callback-style APIs in `src/services/` to async/await.

## Preserve behaviour exactly
- Do NOT change what any function returns or when it throws.
- Do NOT alter error handling semantics. A function that currently
  swallows an error must continue to swallow it — flag it in a
  comment, do not fix it.
- Do NOT change timing or ordering of side effects.
- Do NOT rename anything.
- Do NOT modernise unrelated code you happen to read.

## If you find a bug
Note it in MIGRATION-FINDINGS.md with file and line. Do not fix it
in this change.

## Done when
`pytest -q` passes with zero changes to any test file.

Two clauses do the heavy lifting. Do not fix bugs keeps the diff honest — a migration that also fixes three bugs is unreviewable, because you can no longer tell which behaviour changes were intended. And zero changes to test files is the strongest completion criterion available: if the tests still pass unmodified, behaviour is preserved by definition.

Batching so review stays possible

The reviewability problem is the binding constraint, not the agent's capability. A thousand-file migration in one pull request will be approved without being read.

1. One module per batch. Small enough that a reviewer holds the whole change in their head.
2. Establish the pattern first. Migrate one module, review it properly, agree that this is what the transformation looks like. That reviewed module becomes the reference.
3. Point at the reference. Later batches get "follow the pattern established in src/services/billing/", which is far more effective than describing the pattern in prose.
4. Merge before starting the next. Otherwise later batches branch from a base that does not include the pattern.

Step three is the one that fixes inconsistency. An agent shown a concrete example of the finished transformation produces output that matches it; an agent given a description produces its own interpretation, and the interpretations differ between batches.

Legacy code is not always wrong

The hardest category: code that looks outdated and is deliberate. A synchronous call in an async codebase because the library is not thread-safe. A manual loop instead of a comprehension because it was hot in a profile. A duplicated block because the abstraction leaked.

An agent modernising by pattern cannot distinguish these from ordinary staleness, and will helpfully repair all of them.

# DELIBERATE: synchronous by design. The vendor SDK is not
# thread-safe and async wrappers caused data races (INC-2025-0093).
# Do not convert to async.
def fetch_inventory(sku: str) -> Inventory:
    ...

The comment is the durable fix, because it lives next to the code and survives being read out of context. A rule in your instruction file listing exceptions works only while someone remembers to maintain the list.

Migrations that cannot be batched

Some transformations resist the module-at-a-time approach because the old and new forms cannot coexist — a change to a shared interface, a global configuration switch, a dependency that cannot be installed in two versions at once.

Where you can, buy yourself the ability to batch by introducing a compatibility layer first, as a separate reviewed change:

# src/compat/http.py — TEMPORARY, remove after migration completes
# Lets old callers and new callers coexist during the transition.

from src.adapters.http import Client as _New

class LegacyClient:
    # old call shape, new implementation underneath
    def __init__(self, base_url):
        self._c = _New(base_url=base_url, timeout=30)

    def get_json(self, path, cb):          # old callback style
        cb(self._c.get(path).json())

The shim is throwaway code and worth writing anyway, because it converts one unreviewable change into a sequence of reviewable ones. Give it an explicit removal criterion in the same commit — a temporary layer with no stated end date becomes permanent.

What to check when it is done

• Zero test file changes. Any at all warrants an explanation.
• Consistency across batches. Read three files from different batches side by side, looking for the same decision made differently.
• The findings file. If the agent found no bugs across a large legacy migration, it was not looking — that is a signal the instruction was ignored.
• Deletions specifically. Removed error handling and removed edge-case branches are the migration defects that reach production.