Agent Memory: Four Layers, and Which One Is Worth Building

Every discussion of agent memory is really a discussion of what you choose to write down and where you put it — which makes the real question what deserves carrying at all.

Memory is a design decision, not a feature

Agents are stateless between sessions. Every discussion of "agent memory" is really a discussion of what you choose to write down and where you put it, because nothing persists that you did not persist deliberately.

Framing it that way is more useful than treating memory as something a tool provides, because it makes the real question visible: what is worth carrying, and what should be rediscovered?

Four things people mean by memory

The bottom row is the one to be strict about. Anything a command can answer should be answered by running the command, every time. Caching "there are 47 tests" or "this function takes three arguments" produces confident wrongness within an hour, and it is the failure mode every memory system tends toward.

The layer that actually pays: task handoff

Work that spans sessions is where the absence of memory hurts most, and it is the cheapest layer to build because it is a file.

# Handoff — feature/billing-tiers
Updated: 2026-05-31 by session 4

## Goal
Add per-tier rate limits. Tiers exist; limits do not.

## Done
- `domain/tier.py` — Tier enum, limits table. Tests pass.
- `adapters/redis/ratelimit.py` — token bucket keyed by (tenant, tier).

## Next, in order
1. Wire the limiter into `api/middleware/`. It must run AFTER auth —
   the tier comes from the authenticated principal.
2. 429 response shape: see `contracts/errors.md`, do not invent one.
3. Backfill: existing tenants have no tier. Default to `standard`
   in a migration, do not special-case null at read time.

## Learned, not obvious from the code
- `adapters/redis/` connections are per-request, not pooled. Do not
  hold one across the middleware chain; it will leak.
- `Tier` is serialised into JWTs. Adding a variant is a breaking
  change for tokens already issued. Coordinate before adding one.

## Tried and abandoned
- Limiter before auth: no principal, so no tier. Dead end, do not retry.
- In-process counters: breaks with more than one replica.

The last two sections are what make this worth the five minutes. "Next, in order" is recoverable from the diff; "tried and abandoned" is not, and without it the next session repeats the dead end — sometimes twice.

Where each layer belongs in the tree

AGENTS.md                          committed — conventions, constraints
notes/
  auth-flow.md                     gitignored — subsystem understanding
  handoff-feature-billing.md       gitignored — task state
.agent-logs/                       gitignored — transcripts

The gitignoring is deliberate and worth defending. A committed note becomes documentation somebody feels obliged to maintain, gets reviewed, gets argued about, and goes stale anyway. Keeping it local preserves the property that makes it useful: it is cheap enough to throw away.

The exception is a note that repeatedly proves useful to several people. At that point it has earned promotion into real documentation — but let it earn that rather than assuming it.

Every note needs an expiry

A stale note is worse than none, because the agent trusts it in a way it would not trust a stale README — there is no code in front of it contradicting the claim.

#!/bin/bash
# Warn about notes older than 90 days.
find notes -name '*.md' -mtime +90 -print | while read -r f; do
  echo "stale: $f  (last touched $(date -r "$f" +%Y-%m-%d))"
done

Put the date inside the file as well as relying on mtime, since a checkout resets file times. Two habits keep this honest: date every note in its first line, and delete rather than update when the subsystem it describes has been restructured — a rewritten note tends to keep half the old understanding.

Reading a note back into a session

Persisting the note is half the job. The other half is the agent finding it, which does not happen automatically.

## Notes from previous sessions
- `notes/` holds findings from earlier work — subsystem maps, dead
  ends, handoffs. Check for a relevant one BEFORE exploring.
- Notes are dated in their first line. Anything older than about
  three months may describe code that has since moved; verify
  against the current files before relying on it.
- If a note is wrong, say so and update it. Do not silently work
  around a note that contradicts the code.

The third line matters more than it looks. Without it an agent that notices a note is wrong simply ignores it and carries on, leaving the wrong note in place to mislead the next session. Asking it to flag the contradiction converts a silent decay into a visible one.

What not to build

The elaborate version of this — a vector store of everything the agent has ever learned, retrieved automatically each session — is appealing and usually a mistake at first.

It has no expiry model, so it accumulates contradictions. It retrieves without anyone deciding what is relevant, so it injects yesterday's wrong conclusion alongside today's correct one. And it is opaque: when output degrades, you cannot tell which remembered fragment caused it.

Three markdown files that a person can read and delete outperform it for most teams, and they fail visibly. Build the vector store when you have measured the files failing, not because the files feel unsophisticated.