Test-Driven Loops: Giving an Agent a Definition of Done

Most agent failures come from one root cause: no reliable definition of done. A failing test is the cheapest one available.

Why TDD suits agents unusually well

Most agent failures trace to one root cause: no reliable definition of done. Told to "add a feature", an agent decides for itself when the feature is complete, and its criterion is usually that the code looks finished.

Test-driven development supplies the missing criterion. A failing test is an unambiguous statement of what is required; a passing test is an unambiguous statement that it has been delivered. The loop converges on something checkable rather than on something plausible.

This is why terminal agents paired with a fast test suite outperform the same model given a vague instruction. Nothing about the model changed. The feedback did.

The loop, and who owns each step

Row two is the one to be careful about. Letting an agent write both the test and the implementation removes the independent specification — it will produce a test its implementation passes, which tells you nothing. If the agent writes the test, read it before letting it write the code.

Making the loop runnable

The instruction that turns this from an idea into a workflow:

Work test-first, one behaviour at a time.

For each behaviour:
  1. Write ONE failing test. Run it. Paste the failure output.
  2. Confirm the failure is the expected assertion failure, not an
     import error or a typo. If it is not, fix the test first.
  3. Write the minimum code to pass. Run the test.
  4. Run the full unit suite. Nothing else may break.
  5. Stop and report before starting the next behaviour.

Rules
- Do not modify existing tests.
- Do not write implementation before the test exists and fails.
- If a test needs a change to pass, say so and stop.

Step five is what keeps the whole thing reviewable. Without it the agent runs six cycles and hands you a large diff; with it you get a sequence of small verified steps, and you can stop when the direction is wrong rather than after.

Speed is the binding constraint

The loop runs as many times as the tests are fast. A suite taking two minutes turns a ten-cycle session into twenty minutes of waiting, and the agent starts batching changes to avoid the wait — which destroys the attribution that makes TDD useful.

## Test-driven loop
- Single test:  `pytest path/to/test.py::test_name -x -q`  (~1s)
- Module:       `pytest tests/unit/test_billing.py -q`     (~3s)
- Full unit:    `pytest tests/unit -q`                     (~9s)
- Integration:  `pytest tests/integration -q`  — slow, run once
                at the end, not in the loop

Naming the narrow commands changes behaviour. Given only a whole-suite command, the agent uses it every cycle; given a single-test command, it uses that and the loop tightens by an order of magnitude.

Where the tests should come from

Three arrangements, in descending order of how much they are worth.

Human writes tests, agent implements

The strongest. You have specified behaviour precisely and delegated the tedious part. The test is an independent check the agent cannot bend, because it did not write it.

Agent writes tests from an explicit specification

Workable when the specification is genuinely external — an API contract, a written ticket with acceptance criteria, a bug report with a reproduction. Read the tests before the implementation, always.

Agent writes tests from the existing code

Nearly worthless as verification, and it is what happens by default when you ask an agent to "add tests" to untested code. It describes what the code does today, including its bugs. It has value as a regression harness before a refactor, and none as a statement of correctness — and the difference is worth being explicit about, because the artefacts look identical.

When the loop stalls

Occasionally an agent cycles: change, fail, change, fail, without converging. Three causes, distinguishable by what the failure output says.

None of these are solved by letting it keep trying, and all three burn tokens quickly. A turn limit is worth setting for that reason alone: "if three consecutive attempts fail, stop and describe what you think is wrong" converts an expensive spiral into a useful report.

Refactoring under a green suite

The third step of the cycle is the one teams skip and the one agents are best at. With a passing suite, restructuring is bounded — any behavioural change shows up immediately.

Tests are green. Refactor `OrderService.process` — it is 180 lines.

Constraints
- Run `pytest tests/unit -q` after every extraction. Must stay green.
- Do not change any public signature.
- Do not touch any test file.
- If a change requires a test change, stop and explain why.

Extract one responsibility at a time. Report after each.

The last constraint is the safety property. A refactor that needs a test change is not a refactor — it is a behaviour change wearing a refactor's clothes, and it deserves a conversation rather than a commit.

What the loop does not give you

Passing tests mean the code satisfies the tests. That is all they mean.

• The tests may specify the wrong behaviour. TDD makes an agent build what you asked for, correctly and quickly, including when what you asked for is wrong.
• Whole categories go untested by construction — performance, concurrency, resource cleanup, security properties.
• An implementation can pass every test and still be structurally poor.

So the human review at the end is not ceremonial. What the loop buys is that review starts from working, verified code rather than from a plausible draft, which is a much better place to start.