Context Caching and Reuse: What Survives Between Sessions
Yesterday your agent spent twenty minutes learning how your auth flow works. Today it knows none of it, and will pay to learn it again.
Every session starts from nothing
An agent has no memory between sessions. Yesterday it spent twenty minutes learning that your authentication flow spans four files, that the token refresh lives somewhere unexpected, and that a particular helper is deprecated. Today it knows none of that and will pay to learn it again.
On a large repository this rediscovery is a meaningful share of every session — not the interesting part of the work, just the cost of arriving.
Three things people mean by caching
The word covers three unrelated mechanisms, and conflating them causes confusion about what is possible.
| Mechanism | What it saves | Who controls it |
|---|---|---|
| Provider prompt caching | Cost and latency on repeated prefixes | The provider, with your cooperation |
| Session continuation | Everything — same window | You, by not starting over |
| Externalised notes | Rediscovery across sessions | You, entirely |
Only the third genuinely crosses a session boundary, and it is the one almost nobody sets up.
Where notes should live
A question that decides whether the habit survives: in the repository, or outside it?
In the repository, gitignored is the arrangement that works. The notes sit beside the code so an agent finds them without being pointed at them, and they are not committed — because a note is one person's working understanding at one moment, not a shared artifact anyone has agreed to maintain.
Committing them looks appealing and goes wrong. A committed note becomes documentation people feel obliged to keep current, gets reviewed, gets argued about, and eventually gets stale anyway. Keeping them local preserves the thing that makes them useful: they are cheap enough to throw away.
notes/
.agent-logs/
The exception is a note that has proved durably useful to several people. At that point it has earned promotion into real documentation — but let it earn that rather than assuming it.
Prompt caching: structure the prefix
Providers cache repeated prompt prefixes, so identical leading content on subsequent requests is cheaper and faster. Benefiting from it requires putting stable content first and volatile content last.
STABLE — identical every request, cacheable
system prompt
tool definitions
AGENTS.md
reference files that do not change
VOLATILE — changes every turn, place last
conversation so far
most recent tool results
the current task
The failure is putting anything variable early — a timestamp, a session ID, a randomised ordering of tool definitions. One changed token at the front invalidates everything after it.
If your tool definitions are serialised from a dictionary without a stable sort, their order may vary between runs. That alone can defeat prefix caching entirely, and nothing in the output indicates it is happening.
What the rediscovery actually costs
Worth quantifying, because "it re-reads some files" understates it.
| Session phase | Typical share | Reusable? |
|---|---|---|
| Locating the relevant code | 20–40% | Yes — this is what notes capture |
| Understanding how it fits together | 15–30% | Yes |
| The actual change | 20–40% | No |
| Verifying and fixing | 10–25% | No |
The top two rows are the ones a good note eliminates, and on an unfamiliar subsystem they are the majority of the session. That is the size of the prize, and it explains why the habit pays for itself after roughly one reuse.
Session continuation and its limit
Continuing a session preserves everything, which is why long sessions feel more productive — the agent has accumulated understanding.
The limit is that context degrades as it grows. Early content drifts into the middle, where attention is weakest. Superseded intermediate results accumulate. Failed approaches remain, and the agent sometimes revisits them.
There is a point where a fresh session with a good summary outperforms continuing. Recognising it is a judgement call, and the signal is behavioural: when the agent starts repeating work it did earlier in the same session, the context has stopped helping.
Externalised notes: the part that actually carries over
The only durable mechanism is writing findings down where the next session can read them.
# Auth flow — mapped 2026-08-13
Entry: `api/middleware/auth.py:authenticate()`
→ `domain/auth/token.py:verify()` (JWT validation, no I/O)
→ `adapters/redis/session.py:lookup()` (session lookup)
Refresh is NOT here. It lives in `api/routes/session.py:refresh()`
and does not go through the middleware at all — historical accident.
Gotchas
- `verify()` returns None for both "expired" and "malformed". Callers
cannot distinguish. Bug, not by design.
- Tests mock at the redis layer, so token changes are not covered by
the auth suite. Check `tests/integration/test_session.py` too.
Cheap to produce — ask the agent to write it at the end of a session — and it saves the whole rediscovery cost next time. It is also, not incidentally, the onboarding document no human ever writes.
A stale note is worse than none: it is confidently wrong in the same way a stale README is. Date them, keep them near the code, and delete them when the code moves. Treat them as caches with an expiry, not as documentation.
The summary handoff
When a long session has to end — context is degrading, or the work spans days — the highest-value thing to extract is a structured handoff rather than a chat log.
Before we stop: write a handoff for the next session covering
1. what the task is and what is done so far
2. files changed and why each one
3. what is left, in order
4. anything you learned about this codebase that is not
obvious from reading it
5. approaches you tried that did not work, and why
Write it to notes/handoff-<branch>.md
Point five is the one that repays the effort. Without it the next session repeats the same failed approach, sometimes twice, and there is no way for it to know. A record of dead ends is worth more per line than a record of progress, which is visible in the diff anyway.
What not to cache
Some things should be rediscovered every time, because caching them introduces exactly the staleness this series keeps warning about.
- Current code contents. Read the file. It is the one source that cannot be out of date.
- Test results. A cached green from yesterday says nothing about today.
- Dependency versions. Read the lockfile.
- Anything with a mechanical source. If a command answers the question, run the command.
The distinction that matters: cache understanding, never state. "The auth flow spans these four files and here is why" is understanding, and it stays true for months. "There are 47 tests" is state, and it was wrong an hour later.
Order prompts stable-first so prefix caching works, continue sessions until the agent starts repeating itself, and write findings to dated notes near the code. Cache understanding; never cache state — anything a command can answer should be answered by running the command.