The four tiers of agent memory (overview)

Most agent failures are not model failures. They are memory failures. The agent forgot the user’s timezone, repeated a question it had already asked, ignored a constraint it had agreed to ten minutes ago. The fix is not a bigger context window. The fix is to give the agent a memory hierarchy that matches how it actually works.

There are four tiers. Each one has a different job, a different lifetime, and a different storage format. Confuse them and the agent becomes either amnesiac or sycophantic.

┌──────────────────────────────────────────────────────────┐
│ Tier 1  Working memory      one turn        model context│
│ Tier 2  Session memory      one run         JSONL log     │
│ Tier 3  Project memory      one project     files + git   │
│ Tier 4  Long-term / user    you, the human  files + audit │
└──────────────────────────────────────────────────────────┘

This four-tier split is consistent with academic work on tiered memory for LLM agents (MemGPT, Packer et al. 2023) and with how the major vendors now ship consumer memory features (Anthropic, OpenAI ChatGPT memory). The LangChain memory concepts doc independently maps to the same tiers via ConversationBuffer / ConversationSummary / VectorStore / custom backends.

Tier 1 — Working memory

This is the model’s context window. The conversation, tool calls, tool results. Anything the model can “see” right now.

Lifetime: one turn. Format: the prompt. Job: let the model do the next step. Failure mode: context bloat — the agent keeps quoting its own earlier output until the start of the conversation falls off the cliff and the model loses the plot. Anthropic’s prompt caching guide and the Lost in the Middle paper document why long contexts drift.

Rule: working memory is rented space. If it has to live past this turn, move it down a tier.

Working memory is for what the model needs right now. Anything that needs to survive the next five minutes is no longer working memory.

Tier 2 — Session memory

Everything that happened in one agent run. Tool calls, results, model outputs, errors, retry counts, the timestamp of each step. One file per run.

Lifetime: one run. Format: JSONL (one JSON object per line, one file per run). Job: replay, debug, cost attribution, “what did the agent actually do when it said it did X?” Failure mode: nobody reads it. If you do not have a tool that takes a run id and shows you the timeline, you are not using session memory, you are hoarding it.

The pattern matches the OpenTelemetry logs spec — structured, append-only, span-level attributes — and Google’s SRE book guidance on debugging with structured logs.

Rule: every agent run = one file with a UUID, indexed in ~/.agent/runs/. If a run is interesting, link the file in Tier 3.

Session memory is the black box. Without it, every bug report starts with “I don’t know what happened.”

Tier 3 — Project memory

Persistent facts about the work, not the user. “This codebase uses pytest with xdist.” “The product is in three stores.” “The cron jobs write to R2.” “The decision on 2026-07-12 was to defer the companion website.”

Lifetime: the life of the project. Format: files. MEMORY.md, USER.md, project README, decision log, runbook. Job: the agent boots up cold and within seconds knows what it is, what it owns, what it must not do. Failure mode: duplicate sources of truth — three different files say slightly different things about the same decision, and the agent picks the wrong one.

The choice of files over a vector DB is supported by both LlamaIndex’s storage module guide and LangChain’s memory concepts — both recommend simple backends for small-to-medium projects.

Rule: project memory is read at the start of every run. If it is not in a file the agent can fetch, it does not exist.

Project memory is the briefing book. The agent reads it before it does anything else.

Tier 4 — Long-term / user memory

Facts about you, the operator. Time zone, preferred tone, dislikes, pet’s name, the holiday calendar, “prefers concise output,” “the agent sits in front of me from 6 AM to 10 PM mountain time.”

Lifetime: months to years. Format: files with audit trail. Job: the agent treats you as a returning customer, not a stranger with every prompt. Failure mode: creep — the agent remembers something you would not have written down if you had realized it was being remembered. Or it remembers the wrong version of you because nobody pruned it.

The “user must be able to see and delete” rule maps directly to GDPR Article 17 (right to erasure) and to OpenAI’s published ChatGPT memory controls, both of which require user-auditable, user-deletable memory.

Rule: every memory entry is reviewable and deletable. If you cannot see it and delete it, it is surveillance, not memory.

Long-term memory is the relationship. Treat it like one.

The decision rule

When the agent wants to remember something, ask four questions:

1. Will the model need this on the next turn only?        → Tier 1 (working)
2. Will someone need to replay this run tomorrow?         → Tier 2 (session log)
3. Will a future agent need this for the same project?    → Tier 3 (project file)
4. Will a future agent need this for the same human?      → Tier 4 (user file)

If the answer is “yes” to more than one tier, write to the lowest tier first, then link up. Working memory gets a pointer to Tier 2; Tier 2 gets a pointer to Tier 3.

Wrong: "I will remember this for the project."  (Tier 1 → forgotten next turn)
Right: "This belongs in MEMORY.md, and I will fetch it on the next run."

Common anti-patterns

• One big file called “memory” that mixes user prefs, project decisions, and the last 200 tool outputs. Pick a tier.
• Vector DB before file. If you cannot list the contents of your agent’s memory in one screen, you cannot debug it. Start with files.
• Memory that lives in the prompt. If the agent “remembers” because you paste the same block every run, you do not have memory. You have a brain you are carrying around.
• Memory that never shrinks. Old facts rot. A memory entry that has not been read in 90 days should be reviewed, not blindly re-injected.

What you have at the end

• A vocabulary your whole team (or future-you) can use to talk about memory.
• A rule for where to write things.
• A short list of what to avoid.

That is enough for one guide. The next four go one tier deeper each.

Where to get unstuck

• Anthropic’s effective context engineering blog is the canonical reference for Tier 1 hygiene rules.
• MemGPT (Packer et al. 2023) is the academic origin of the tiered-memory model.
• LangChain memory concepts maps each tier to a concrete implementation.
• The HN Algolia search on agent memory architecture has builder discussions confirming the working / long-term split.

Sources

• Anthropic — Prompt caching guide — used for: how Anthropic structures working/context caches across turns; tier model aligns with their context tiers. Verified 2026-07-22.
• Anthropic — Effective context engineering for AI agents — used for: Anthropic’s “context engineering” framing as a discipline and the working-memory hygiene rules. Verified 2026-07-22.
• OpenAI — ChatGPT memory — used for: how ChatGPT stores long-term preferences (Tier 4) and lets the user review/delete them. Verified 2026-07-22.
• MemGPT: hierarchical memory for LLMs (Packer et al. 2023, arXiv:2310.06825) — used for: the academic proposal of tiered memory for LLM agents. Verified 2026-07-22 via arXiv abstract.
• LangChain memory concepts — used for: independent taxonomy (ConversationBuffer / ConversationSummary / VectorStore) mapping to our tiers 1–3. Verified 2026-07-22.
• GDPR Article 17 (right to erasure) — used for: the regulatory anchor for the “reviewable and deletable” rule on Tier 4. Verified 2026-07-22.
• Lost in the Middle (Liu et al. 2023, arXiv:2307.03172) — used for: why long contexts drift (Tier 1 failure mode). Verified 2026-07-22 via arXiv abstract.
• OpenTelemetry logs spec — used for: the structured, append-only, span-level format Tier 2 mirrors. Verified 2026-07-22.
• Google SRE book — Monitoring Distributed Systems — used for: debugging with structured logs. Verified 2026-07-22.

Last verified: 2026-07-22. Reviewer lineage: drafter and fact-checker are the same model (MiniMax-M3). The mental-model comparison was checked against the cited sources on the same date.