Reviewing Agent-Authored Code: The Author Cannot Answer Questions

You learned to skim tidy code and slow down at messy code. Agent output is uniformly tidy, which means that instinct now sends you past everything at speed.

Reviewing without an author

Code review assumes an author. You ask why they chose an approach, what they considered, whether they thought about the edge case. That conversation is most of the value — the defects it surfaces are usually in the reasoning rather than the syntax.

Agent-authored code removes it. The reasoning existed in a context window that has been discarded. Nobody can answer "why did you do it this way", because nobody decided.

Reviewing it as though a colleague wrote it therefore misses systematically, because the failure modes are different ones.

The volume problem

Before the specifics, the structural issue: agents shift the ratio of code written to code reviewed. One engineer with an agent can produce several changes in the time they previously produced one. Review capacity did not change. Whatever your team's review throughput was last year, it is roughly the same now, and more is arriving at it.

Two failure modes follow, and teams tend to pick one without deciding to. Review becomes a bottleneck and changes queue, which is visible and annoying but not dangerous. Or review quality quietly degrades to keep pace, which is invisible and is how defects ship.

The second is worth watching for explicitly, because nobody announces it. The signal is approval latency dropping while change volume rises — if PRs are being approved faster than they used to be while more of them arrive, something has given, and it was not anyone's ability to read code quickly.

What differs, specifically

Row two is the trap. Agent code reads well. Sensible names, consistent formatting, explanatory comments, a plausible structure. Surface quality is normally a decent proxy for care taken, and here it is uncorrelated — which means the signal reviewers rely on to allocate attention has been severed.

Five questions that actually find things

Does the test test the right thing?

The most common substantive defect. An agent asked to fix a bug and add a test will sometimes write a test asserting what the code now does, rather than what it should do. It passes. It would pass against the bug too, if the bug were reintroduced differently.

Check it directly: revert the fix, keep the test, confirm it fails. If it does not, the test is decorative.

Is anything here outside what was asked?

Agents expand scope. A request to fix one function comes back with three other functions tidied, a helper extracted, and some imports reordered. Each change is defensible. Collectively they turn a two-line review into a forty-line one and bury the actual fix.

Ask for it to be split. This is more important with agent output than human output, because there is no author whose judgement about relatedness you can lean on.

Does it match how this codebase does things?

Agents follow patterns from what they have read this session plus general priors. If the relevant files were not open, the priors win, and you get an idiomatic implementation of a pattern your codebase does not use.

What is the error handling doing?

Consistently the weakest part. Look for swallowed exceptions, logged-and -continued failures that should propagate, and retries around operations that are not idempotent.

Was anything deleted?

Deletions get less scrutiny than additions in every review tool, and an agent that removed a check it did not understand produces a diff that looks smaller and simpler. Read the removed lines specifically.

The consistency drift

One defect deserves separate treatment because reviewers reliably miss it: consistency within a change.

An agent working across many files over a long session does not hold the whole change in view. Early files reflect one set of decisions; later files reflect decisions made after reading more code. The result is a diff that is locally sensible everywhere and globally incoherent.

src/billing/invoice.py     raises InvoiceError on failure
src/billing/payment.py     returns Result[Payment, str]
src/billing/refund.py      returns None and logs

Three error-handling conventions in one pull request. Each file reads fine alone; nobody reviewing file by file notices. The fix is to read the diff looking specifically for the same decision made differently in different places — error handling, naming, return shapes, logging. It is a different reading mode from checking each hunk, and it is the one that catches this.

Making the PR reviewable

Most of the difficulty is fixable at authoring time rather than review time.

• Include the prompt. Reviewers cannot judge whether the change answers the question without knowing the question.
• Label it. Reviewers should know to calibrate differently.
• Keep them small. The scope-expansion tendency makes this harder and more necessary.
• Attach the transcript where the tooling allows. It contains the reasoning that would otherwise be lost, including approaches tried and abandoned.

Calibrating by task type

Not all agent output warrants equal scrutiny, and pretending otherwise exhausts reviewers on low-risk changes and leaves nothing for high-risk ones.

Row one is worth stating explicitly because reviewers often over-invest there — a four-hundred-file rename feels alarming and is one of the safest things an agent does, since the type checker verifies every site. Save the attention for row four, where nothing verifies the decisions.

The question nobody wants to answer

Who is accountable for agent-authored code that causes an incident?

The only workable answer is the person who opened the pull request. They ran the agent, they read the output, they asked for it to be merged. "The agent wrote it" describes how the text was produced and settles nothing about responsibility.

Worth stating explicitly as team policy, because the alternative is a diffusion where nobody feels ownership of code everybody uses. Teams that write this down review agent output more carefully, which is the point.