Tier 1 working memory: prompt hygiene beats size

Working memory is the model’s context window. It is rented space. Every token you put in there costs you latency, money, and a non-zero probability that something important falls out the other side.

The naive move is to “add more context.” The right move is to keep the prompt small and shaped so the model can see what matters. Both Anthropic and OpenAI now publish prompt caching guidance built around the same idea: stable prefixes get cached, mutable content lives at the edges.

Where this comes from: Marvin’s 2026-07-20 original. Sources verified 2026-07-22 against Anthropic, OpenAI, the cited arXiv papers, and LangChain memory docs.

The five rules

1. Recency is not enough

A 100k context window does not mean the model is paying attention to all 100k tokens. Attention is heaviest near the prompt edges (primacy + recency). The middle fades.

This is the empirical finding of “Lost in the Middle” — Liu et al. 2023: when relevant information is buried in the middle of a long context, retrieval accuracy drops materially compared with putting it at the start or end. Anthropic’s prompt caching guide and the Attention Sinks paper (Xiao et al. 2023) explain why: attention pools at the start of the sequence, so a few initial tokens matter more than their length suggests.

Strongly attended    Strongly attended
       ┌──────────────────┐
       │  system │ user-1 │ … │ user-N │
       └──────────────────┘
       ↑                              ↑
    first 5%                       last 5%

If something is load-bearing, put it at the top or the bottom. Do not bury it.

2. Compress, do not copy

When the conversation gets long, summarize earlier turns into a short TL;DR block, do not re-paste them. The model can re-derive from a TL;DR faster than it can re-read 30k tokens of stale assistant monologue.

# Earlier turns (compressed)
- User asked to schedule 7 social posts across 3 stores for tomorrow.
- I drafted titles and tags, got approval, queued to pending-posts.json.
- ...

LangChain’s ConversationSummaryMemory implements exactly this pattern: maintain a rolling summary, drop the original transcript.

3. De-duplicate by reference

If a tool result is large and you need to refer back to it, store it in Tier 2 (session log) and reference it by id. Do not paste it into the conversation a second time.

# In prompt
Receipt OCR result: see run 7c2a, item #4 ("...the vendor name was truncated")

# In session log (Tier 2)
run_id=7c2a  step=4  tool=ocr.vision  input=receipt.jpg  output=...

4. Strip the system preamble

If the same 400-token system message is sent on every turn, you are paying for it 50 times in a 50-turn run. Move stable instructions to Tier 3 (project memory) and inject them once at session start, not on every turn. Anthropic’s prompt caching guide is explicit: caches are most effective when the prefix is stable across requests, so a static system prompt is the ideal cache candidate.

5. Drop successful retries

When a tool fails twice and succeeds on the third try, the conversation does not need the first two failures. Compress to: “Tool X failed twice with E, succeeded on attempt 3.” That keeps the cause-of-failure visible without the noise.

The shape of a healthy working memory

[ system — short, stable, project-specific, ~150 tokens ]
[ project memory pointer — "see MEMORY.md" + 5 most relevant lines, ~200 tokens ]
[ compressed earlier-turns block, scales with run length, capped at ~500 tokens ]
[ tool results — current turn only, references for old ones ]
[ user message — current turn ]

That is roughly 1,000 tokens of stable framing + whatever the current turn needs. Anything that has to live past this turn moves to Tier 2 or Tier 3.

Failure mode: the cliff

If you do nothing about context growth, the agent will eventually hit a wall. Symptoms:

• Model starts ignoring instructions from the start of the conversation.
• Model invents things it was told earlier.
• Cost and latency climb linearly with conversation length.
• A token-budget error fires at turn 47.

The fix is the same every time: stop putting working memory where Tier 2 or Tier 3 belongs. There is no prompt hack that beats that.

Anti-patterns

• The forever scroll. Agent and user take turns for an hour. Nobody summarizes.
• The tool result sandwich. Agent pastes full tool output into every follow-up turn.
• The pinned block. 800-token system message that never changes, sent every turn, paid for 50 times.
• The panic dump. When a tool fails, the agent dumps the entire previous conversation “just to be safe.”

Trade-offs (per tip)

Tip	What it costs	What it saves
Compress don’t copy	A summary call on long runs (every N turns).	Re-reading 30k tokens of stale monologue = saves 30x context.
De-duplicate by reference	Tier 2 lookup latency (seconds).	Pasted tool results on every turn = context bloat.
Strip the system preamble	Tier 3 injection latency at session start.	Same 400 tokens × 50 turns = 20k tokens saved per run.
Drop successful retries	Loss of failure context (rarely valuable).	Two pasted failed tool outputs = 5k tokens saved.

When these tips stop applying

• Single-turn tasks. If your agent never has a conversation (one prompt, one response, exit), Tier 1 hygiene doesn’t matter. The conversation has one turn.
• Sub-1k-token prompts. The hygiene overhead is overhead; for tiny tasks the cost exceeds the savings.
• Models with infinite context and infinite attention. Not a thing in 2026.
• When the agent owns its own memory system. If the framework manages context for you (LangChain’s auto-summarization, Hermes’s built-in memory), these are framework defaults — check the docs before layering your own rules.

What you will have at the end

• Five rules for keeping working memory small.
• A template for the prompt shape that survives long runs.
• A diagnostic for the cliff failure mode, grounded in the Lost-in-the-Middle finding.

Where to get unstuck

• Anthropic prompt caching for the cost side: docs.
• OpenAI prompt caching for the cost side: guide.
• Anthropic effective context engineering for the framing side: article.
• Lost in the Middle for the empirical “attention is not uniform” finding: Liu et al. 2023, arXiv:2307.03172.
• Attention Sinks for why the first tokens matter so much: Xiao et al. 2023, arXiv:2309.17453.
• LangChain memory concepts for the framework-side implementation: memory docs.

Sources

• Anthropic prompt caching — used for: the stable-prefix caching framing, the rationale for static system prompts. Verified 2026-07-22.
• OpenAI prompt caching guide — used for: the OpenAI parallel, stable-prefix reasoning. Verified 2026-07-22.
• Anthropic effective context engineering — used for: the “context engineering” framing and the structural advice. Verified 2026-07-22.
• “Lost in the Middle” — Liu et al. 2023 — used for: the empirical basis for the recency-not-enough rule. Verified 2026-07-22.
• Attention Sinks — Xiao et al. 2023 — used for: the explanation of why the start of the sequence matters more than its length. Verified 2026-07-22.
• LangChain memory concepts — used for: ConversationSummaryMemory as the framework implementation of “compress don’t copy.” Verified 2026-07-22.

Last verified: 2026-07-22. Adapted from Marvin’s original “B2. Tier 1 — Working Memory: Prompt Hygiene Beats Size” (2026-07-20). Drafter and source-notes author are the same model (MiniMax-M3). Tips category — no subagent review required per the v2.4.0 skill rule (Setup/Playbook only).