Build System Integration: Your Toolchain Is the Agent's Feedback Loop

A test runner that exits 0 on failure means the agent's only verification signal is lying to it. Everything after that is built on a false green.

The build system is the agent's feedback loop

An agent's ability to verify its own work is bounded by how well it can run your build. If the commands are slow, unreliable or produce output that is hard to interpret, the agent operates with degraded feedback — and degraded feedback shows up as degraded output, attributed to the model.

This makes build system quality an unexpectedly high-leverage place to invest. The same properties that make a build pleasant for humans make it usable for agents, and the agent is a much less forgiving consumer.

Four properties that matter

Fast enough to run repeatedly

A fourteen-minute test suite cannot be part of an iteration loop. The agent runs it once, waits, and then makes several changes before running it again — which is exactly the batching that makes failures hard to attribute.

You need a fast subset. Under thirty seconds, ideally under ten. It does not have to be complete; it has to be fast enough that running it after every change is free.

Deterministic

Flaky tests are worse for agents than for humans. A human sees a familiar flake and re-runs. An agent sees a failure, believes it, and starts fixing code that was never broken — sometimes producing a real bug in pursuit of a phantom one.

If you cannot fix the flakes, name them in your context file so the agent knows to re-run rather than investigate.

Machine-readable failure output

Compiler and test output is written for humans. Some of it parses cleanly; some does not.

Most tools have a machine-readable mode that nobody uses interactively. Putting the flag in your context file costs one line and materially improves what the agent does with a failure.

## Verification
- `pytest -q --tb=line` — one line per failure, not full tracebacks
- `mypy src/ --no-error-summary --no-pretty` — parseable positions
- `eslint . -f compact` — file:line:col format

One entry point

A single command that runs everything CI runs, in the same order, failing at the first problem.

#!/bin/bash
# Everything a PR must pass. CI runs exactly this.
set -e
ruff check .
ruff format --check .
mypy src/ --no-error-summary
pytest tests/unit -q --tb=line

Three benefits. The agent has an unambiguous definition of done. The instruction cannot go stale, because if check.sh breaks, CI breaks and someone fixes it. And there is exactly one thing to name in your context file instead of four.

Where builds break agents specifically

Row three is the worst and the least visible. A test runner that exits 0 on failure — through a swallowed error, a misconfigured reporter, a || true someone added years ago — means the agent's only verification signal is lying to it. Everything downstream is built on a false green.

Worth checking directly, right now: break a test deliberately, run your check command, and confirm the exit code is non-zero. It takes a minute and it occasionally finds something alarming.

Timeouts and long-running commands

Agents have finite patience — a command that produces no output for several minutes may be judged hung and killed, even when it is working correctly. A twelve-minute test suite that prints nothing until the end is particularly prone to this.

Two fixes, both small. Make long commands report progress, so there is evidence of life. And tell the agent what to expect:

## Timing
- `pytest tests/unit` ~8s
- `pytest tests/integration` ~4 min, silent for the first 90s
  while containers start. This is normal. Do not kill it.
- `npm run build` ~2 min. Prints nothing until finished.

Naming the silent periods prevents a specific and wasteful failure: an agent killing a working build, concluding the build is broken, and starting to investigate a problem that does not exist.

Test selection changes what the agent tries

A subtle effect worth knowing: how easy it is to run a subset of tests shapes how the agent works.

If running one test file is easy, the agent iterates tightly — change, run the relevant test, adjust. If the only option is the whole suite, it batches several changes between runs, and a failure then implicates all of them at once. The debugging that follows is slower and less reliable.

## Running a subset
- One file:     `pytest tests/unit/test_billing.py -q`
- One test:     `pytest tests/unit/test_billing.py::test_refund -q`
- By keyword:   `pytest -k "refund and not integration" -q`
Use the narrowest scope that covers your change while iterating.

Three lines, and they change the shape of the work rather than merely informing it — which is a useful reminder that a context file is not only documentation, it is a set of affordances.

Preconditions belong in the script

Rather than documenting that integration tests need Docker, have the script check and say so:

#!/bin/bash
set -e
if ! docker info >/dev/null 2>&1; then
  echo "ERROR: Docker is not running. Start it, then re-run." >&2
  exit 1
fi
if ! docker compose ps --status running | grep -q postgres; then
  echo "Starting dependencies..." >&2
  docker compose up -d --wait
fi
pytest tests/integration -q --tb=line

The difference in agent behaviour is large. Without the check, a missing Docker daemon produces connection timeouts sixty seconds later, and the agent starts investigating the database client. With it, the failure is immediate and names its own remedy.

This is the recurring theme of the whole pillar: every ambiguity you remove from your tooling is an ambiguity the agent cannot resolve wrongly. And the same change makes the build better for the humans who have been quietly working around it.