Local Execution: It Worked in the Session Is the New Works on My Machine

The transcript says the tests passed. It does not say which Node version ran them, or that a shell function stopped the suite at the first failure.

The security question has an answer; the correctness question does not

An agent running in your own terminal reads your files, runs your toolchain, and touches your services. The security consequences of that are well covered — the machine is the boundary, and shell execution blast radius works through what that means and how to contain it.

This article is about the other half, which gets far less attention and causes more day-to-day trouble: local execution makes results depend on the machine they were produced on. "It worked in the session" is the new "works on my machine", and it is harder to debug, because the session is gone and nobody recorded what the environment looked like while it ran.

What ambient means here

Every command an agent runs resolves against state nobody passed to it. The PATH that decides which python answers. The virtual environment that happens to be active. Environment variables set by a shell profile three years ago. A .env file loaded automatically by a directory hook. Version managers that switch the toolchain based on the directory. A cloud CLI already authenticated against a particular project.

None of that is visible in the transcript. The transcript says npm test and all 214 passing, and both are true, and neither tells you which Node version ran them.

The failure looks like flakiness

The characteristic shape: an agent makes a change, runs the test suite, sees green, and reports success. CI then fails on something unrelated-looking — a date formatting difference, a missing native dependency, a test that assumes a locale. Everyone assumes CI is flaky. It is not; it is the only participant with a declared environment.

The reason this is worth naming is that the natural response is to distrust the agent's work in general, which is both demoralising and wrong. The work was fine. The verification ran somewhere undeclared.

Shell configuration is instruction surface

This one surprises people. Your shell's aliases and functions rewrite commands before they execute, so an agent that runs git may not be running git.

# Innocuous personal conveniences that alter agent behaviour:
alias git='git --no-pager'          # changes output the agent parses
alias rm='rm -i'                    # prompts, and the agent has no tty
alias python=python3.11             # not the project's interpreter

pytest() { command pytest -x "$@"; }   # stops at first failure, so the
                                        # agent sees one error, not twelve

The pytest wrapper is my favourite example because it is actively helpful for a human and actively misleading for an agent: the agent concludes there is one failing test, fixes it, re-runs, finds another, and works through twelve failures one round trip at a time instead of seeing them together.

Interactive prompts have no answer

A command that waits for confirmation in an agent session either hangs until a timeout or receives whatever the harness sends. Both outcomes are bad, and the second is worse because it looks like it worked.

The general fix is to make non-interactivity explicit rather than accidental: CI=true, DEBIAN_FRONTEND=noninteractive, --yes flags where the semantics are safe, and a GIT_TERMINAL_PROMPT=0 so a credential prompt fails fast instead of hanging. Set them in the session's environment rather than hoping every command is invoked correctly.

State accumulates across a long session

A container run starts clean every time. A local session does not. Two hours in, the working tree has scratch files, the database has rows from four test runs, a background process from a debugging detour is still holding a port, a global package got installed to make one thing work, and the branch is three commits into something abandoned.

None of that is a bug and all of it changes behaviour. The agent's model of the repository is also now partly wrong, because it has been reasoning from what it wrote rather than what is on disk. The cheap correction is a periodic git status and a look at what is listening, not because either is interesting but because divergence between the assumed and actual state is what produces confidently wrong next steps.

Declare the environment where the work happens

The whole of the fix is one idea: move the environment from the machine into the repository, so it travels with the code and can be asserted rather than assumed.

A pinned toolchain version in a file the version manager reads. A lockfile that is actually respected — npm ci rather than npm install, uv sync --frozen rather than a bare install. Explicit locale and timezone for anything that formats. A single documented command that runs the tests the way CI runs them, so there is one answer rather than a per-developer convention.

#!/usr/bin/env bash
# Print the environment a result was produced in. Cheap, and it turns
# "it worked in the session" into something checkable.
{
  echo "date:      $(date -u +%FT%TZ)"
  echo "host:      $(uname -srm)"
  echo "shell:     $SHELL"
  echo "cwd:       $PWD"
  echo "git:       $(git rev-parse --short HEAD) on $(git rev-parse --abbrev-ref HEAD)"
  echo "dirty:     $(git status --porcelain | wc -l | tr -d ' ') files"
  echo "node:      $(command -v node) $(node --version 2>/dev/null)"
  echo "python:    $(command -v python) $(python --version 2>&1)"
  echo "venv:      ${VIRTUAL_ENV:-none}"
  echo "TZ/LANG:   ${TZ:-unset} / ${LANG:-unset}"
  echo "aliases:   $(alias | wc -l | tr -d ' ') defined"
} | tee .last-env-report

Have the agent run it before a test run whose result will be reported to anyone else. The output is twelve lines and it converts a claim into evidence.

Reproducible enough, not reproducible

Full reproducibility is a container, and containers cost you the thing local execution was for: your running services, your authenticated CLIs, your editor state, your actual database. That trade is examined in sandboxing agent environments, and for a lot of day-to-day work the container is the wrong side of it.

The middle position is to make the parts that affect results declared, and leave the rest ambient. Toolchain versions, dependency resolution, locale and the test command: declared. Which terminal emulator you use, where your editor puts swap files, your shell prompt: ambient, and irrelevant.

The one-command rule

If there is a single habit worth adopting from this article, it is that every verifiable claim should correspond to one command that anyone can run. Not a sequence the agent worked out, not "run the tests but skip the slow ones", not a flag someone remembers. One command, in the repository, used by the agent and by CI.

That single change removes most of the divergence, because the agent stops inventing invocations. It is also the thing that makes an agent's verification worth anything: a result produced by the same command the pipeline runs is evidence about the pipeline, and a result produced by a bespoke invocation is evidence about that invocation. The general argument is in build and test instructions for agents, and reproducibility is the reason it pays off.

Where local execution is straightforwardly better

None of this is an argument against working locally. Debugging something that only reproduces against your real database. Iterating with a dev server already running and warm. Anything involving hardware, a VPN, an internal service, or a dataset too large to sync. Work where the feedback loop is seconds and a container round trip would be minutes.

In those cases the ambient environment is exactly the asset. The point is to know which of your results depend on it, so that the ones you hand to other people do not.

What to change on a normal laptop

Pin the toolchain in the repository. Add the one test command. Put CI=true and a couple of non-interactive variables in the agent's environment. Check type -a on your main tools once, and move any alias that rewrites behaviour into a differently named function. Have the agent print the environment report before reporting a result to anyone else.

That is an hour of work, it does not restrict what the agent can do, and it converts the most common false-confidence failure — green locally, red in CI, blamed on flakiness — into something with a visible cause.

The agent's memory of the repository goes stale too

There is a second kind of drift in a long local session, and it is between the agent's model of the code and the code. The agent read config.py ninety minutes ago. Since then you edited it in your editor, a rebase moved three files, and a dependency update rewrote a lockfile. None of that entered the session.

The symptom is a confidently wrong edit: a change applied to a function signature that no longer exists, or a patch that conflicts with something the agent has no idea happened. It reads as the agent being unreliable, and it is really an out-of-date snapshot being treated as current.

The habit that fixes it is cheap: before a substantial edit, re-read the file rather than working from what the session remembers, and check git status and git log --oneline -3 at the start of any resumed session. Both are one line. Neither is interesting until the session it saves.

Background processes are the ones that bite later

An agent that starts a dev server, a watcher, a tunnel or a database container during a session frequently does not stop it. The process outlives the session, holds a port, keeps a file lock, and continues writing to a log until something else fails for a reason that looks unrelated.

The classic version is a stale server on the port your tests expect. The next run passes against the old build, and the agent reports that its change works. That is a false green produced entirely by leftover state, and it is essentially undiagnosable from the transcript.

Ask for backgrounded processes to be started with an explicit way to stop them, and check for strays at the end of a session — lsof -i -P -n | grep LISTEN on a Mac, or the equivalent, takes a second and occasionally explains a whole afternoon.

Two machines, two answers

The clearest demonstration of the whole problem: give the same task to the same agent on two developers' laptops and compare. Different toolchain versions resolve, different services are running, different aliases apply, different credentials are live. The transcripts will look similar and the results will not always agree.

Teams discover this when someone says a task "doesn't work for me" and nobody can reproduce the discrepancy, because reproducing it means comparing two undeclared environments. The environment report from earlier turns that comparison into a diff of twelve lines, which is a solvable problem rather than an argument about whose setup is wrong.

What this means for trusting an agent's verification

The underlying point is about what a green result is evidence for. An agent running the test suite in a declared environment produces evidence about the code. The same agent running an ambiguous command in an undeclared environment produces evidence about that moment on that machine, which may or may not generalise.

Both look identical in the transcript — a command, some output, a summary. So the trustworthiness of an agent's self-verification is not a property of the agent at all; it is a property of how well the environment it ran in was pinned down. Teams that find agent verification unreliable have usually not made that distinction, and the fix is in the repository rather than in the prompt.