Structural Refactoring: Have the Agent Write the Transformation, Not the Edits

A four-hundred-file diff gets approved without being read. A twelve-line query plus a spot-check gets reviewed properly — and the query is where the mistake would be.

The refactor an agent cannot be trusted with

Renaming a symbol across four hundred files is mechanically simple and semantically dangerous. Simple because the transformation is well defined. Dangerous because an agent doing it by reading and editing will occasionally rename a string literal, miss a dynamic reference, or helpfully improve something adjacent.

The resolution is not to trust the agent more carefully. It is to change what the agent produces: instead of edits, have it produce a transformation that a parser applies deterministically.

Why tree-sitter specifically

Tree-sitter parses source into a concrete syntax tree, incrementally, across most languages, with error recovery that keeps working on files that do not compile. That last property matters more than it sounds — mid-refactor code frequently does not compile, and a parser that gives up on broken input is useless for exactly the moment you need it.

Row three is the one that matters for review. A four-hundred-file diff gets approved without being read. A twelve-line query plus a spot-check of the output gets genuinely reviewed, and the query is where a mistake would be.

Queries, briefly

Tree-sitter queries are s-expressions matching tree shapes, with @captures naming the nodes you want back.

; every function definition's name
(function_definition name: (identifier) @fn_name)

; calls to a specific method on any object
(call
  function: (attribute
    object: (_) @obj
    attribute: (identifier) @method)
  (#eq? @method "execute"))

; except-clauses that do nothing — swallowed exceptions
(except_clause
  body: (block (pass_statement) @swallowed))

The third is a good example of what this buys. "Find every place we swallow an exception" is not a text search — pass appears everywhere — but it is a two-line structural query.

Verify the query before trusting it

A query that silently matches the wrong set is the failure mode here, and it is easy to check.

# List matches with context, change nothing
python3 apply.py --dry-run src/ | tee matches.txt
wc -l matches.txt

# Sanity: does the total match what you expect?
rg -c 'datetime\.now' src/ | awk -F: '{s+=$2} END {print s}'

The second command is a deliberately crude cross-check. If grep finds 340 occurrences and your structural query matched 290, the difference is worth understanding — it may be comments and strings, which is correct, or it may be a call shape your query missed, which is not.

The workflow that holds

1. SURVEY   agent writes a query; you run it; review the match list
2. REVIEW   check the matches are the right set — no more, no fewer
3. APPLY    a script transforms every match deterministically
4. VERIFY   type checker and tests confirm nothing broke

Stage two is the human checkpoint, and it is cheap because you are reading a list of locations rather than a diff. If the query matches 412 sites and you expected roughly 400, that is worth a look before anything changes. If it matches 38, the query is wrong and you have found out before touching the codebase.

import sys
from tree_sitter import Parser
from tree_sitter_languages import get_language, get_parser

LANG = get_language("python")
parser = get_parser("python")

QUERY = LANG.query('''
(call
  function: (attribute
    object: (identifier) @obj
    attribute: (identifier) @meth)
  (#eq? @obj "datetime")
  (#eq? @meth "now")) @call
''')

def rewrite(path: str) -> int:
    src = open(path, "rb").read()
    tree = parser.parse(src)
    spans = [n.byte_range for n, cap in QUERY.captures(tree.root_node)
             if cap == "call"]
    if not spans:
        return 0
    out = bytearray(src)
    # apply back-to-front so earlier offsets stay valid
    for start, end in sorted(spans, reverse=True):
        out[start:end] = b"clock.now()"
    open(path, "wb").write(bytes(out))
    return len(spans)

total = sum(rewrite(p) for p in sys.argv[1:])
print(f"rewrote {total} call sites")

Where the agent adds value

Writing tree-sitter queries by hand is fiddly — the grammar node names differ per language and are not memorable. This is precisely what an agent is good at: give it an example of the code you want to match and ask for a query that matches it.

Write a tree-sitter query for Python matching every call to
`session.execute()` where the argument is an f-string.

Then write the inverse: the same call where the argument is NOT
an f-string, so I can confirm the counts add up to all calls.

Do not modify any files. Output only the two queries.

The second query is the check. Two complementary queries whose counts sum to the total tells you the first one is not silently missing a case — a verification you cannot easily perform on a diff.

When to stop and use a real tool

Tree-sitter is the right level for transformations that are structural but not type-dependent. Above that line, reach for the language's own refactoring tooling.

The mistake is reaching for the general tool when a precise one exists. An LSP rename in a typed language is exact in a way no syntactic match can be, because it resolves what the symbol actually refers to rather than what it is spelled.

What this does not cover

Structural matching is syntactic. It does not know types, does not resolve imports, and cannot follow dynamic dispatch.

• A rename matching process() hits every class's process, not only the one you meant. Narrow the query by the enclosing class, or accept a review pass.
• Aliased imports defeat naive matching — from datetime import datetime as dt produces dt.now(), which the query above misses entirely.
• Anything reached through getattr, reflection or a string-keyed dispatch table is invisible.

The mitigation is the verify stage rather than a cleverer query. A type checker catches the rename that hit the wrong class; a test suite catches the alias you missed. The transformation being deterministic is what makes those signals meaningful — you know exactly what changed, so a failure points at the query rather than at a hundred independent edits.