guide · ai

Context Engineering

Context engineering: curating exactly what an AI agent sees, the discipline named in mid-2025.

August 12, 2026 · By Alastair Fraser

A retro robot sorting floating document cards, keeping some and discarding others

The problem

AI agents keep failing for boring reasons. They forget what they said ten minutes ago. They contradict a fact they wrote down themselves. They burn through their working memory mid-task and start hallucinating. The naive fix — “give the model a longer prompt, or a longer conversation history” — stops working past a certain size, because the model’s attention has a budget just like yours does. Something has to decide what makes it into that budget. That something is context engineering.

What it is

Context engineering is the discipline of assembling, at every step of an AI’s work, exactly the right information, tools, examples, and instructions — in exactly the right format — so the model has what it needs to get the task done. Where prompt engineering optimizes how a single instruction is worded, context engineering optimizes the entire configuration of tokens the model sees: system prompt, retrieved documents, tool definitions, prior turns, scratch notes, and external state.

The term got named in mid-2025. Shopify CEO Tobi Lütke tweeted on June 19, 2025 that he preferred “context engineering” over “prompt engineering” because it “describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.” Four days later, LangChain’s Harrison Chase formalized the term in “The rise of ‘context engineering’”, and Anthropic shipped the canonical practitioner deep-dive, “Effective context engineering for AI agents,” on September 29, 2025.

The concept wasn’t invented in 2025. It descends from Retrieval-Augmented Generation (RAG) — the early-2020s pattern of fetching relevant documents into the prompt — and from years of in-context learning research. What changed in 2024–2025 was the arrival of agents that run for hours, not turns. Once a model has to make dozens or thousands of tool calls in a row, “write a clever prompt” stops being the bottleneck. Curating the inputs becomes the whole job.

How it works

Think of the model’s context window as a desk. There’s only so much room on it. Context engineering is the art of deciding what goes on the desk, what’s filed away, and what’s thrown out. The core techniques:

Compaction. When the conversation gets near the desk’s edge, summarize what’s already happened and start a fresh desk with the summary on it. Preserves architectural decisions and unresolved details; discards redundant tool output. Used in long tasks — codebase migrations, multi-hour research.
Notes and memory. The agent regularly writes notes to a persistent file (a NOTES.md, a /memories directory) and reads them back when relevant. Claude playing Pokémon for thousands of steps uses this trick to track goals across days of play.
Just-in-time retrieval. Don’t load the whole library onto the desk. Keep lightweight pointers in context — file paths, URLs, saved queries — and let the agent fetch the actual data only when needed. Mirrors how humans use bookmarks, not memorization.
Sub-agents. A lead agent holds the high-level plan and dispatches focused tasks to sub-agents, each with their own clean desk. Sub-agents explore extensively but return only a distilled summary — typically 1,000–2,000 tokens — back to the lead.
KV-cache optimization. A production-grade lever most non-engineers never hear about. Models cache the math they did on a stable prompt prefix, and cached input tokens cost roughly one-tenth as much as uncached ones. A single timestamp or non-deterministic JSON field can silently invalidate the cache and 10x your bill.

As Anthropic’s team put it, “Good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome”. The bar isn’t “more context.” It’s less, but better.

A few practitioner habits the canonical sources converge on:

Start with the smallest viable prompt. Test the minimal system prompt against the strongest model; only add instructions and examples when a clear failure mode shows up. Resist the urge to over-write.
Hit the Goldilocks altitude. Specific enough to guide behavior, flexible enough to give the model strong heuristics. Avoid both brittle hardcoded if-else logic and vague hand-wavy guidance — either extreme wastes context.
Structure with tags and headers. Use XML tags or Markdown headers (<background_information>, <instructions>, ## Tool guidance, ## Output description) to delineate sections. The model parses them more reliably than walls of prose.
Curate examples, don’t enumerate them. A few diverse, canonical few-shot examples beat a sprawling catalog of edge cases. For LLMs, examples are the pictures worth a thousand words.
Engineer for the KV-cache. Keep the prompt prefix stable, keep message history append-only, ensure deterministic serialization, mark cache breakpoints explicitly. This step alone can mean the difference between a viable product and an unviable one.

Anthropic’s team found that for production agents, spend more time on tool design than on the prompt itself. A well-designed tool is self-contained, robust, unambiguous, and poka-yoked against misuse. A bad tool forces the model into long error-recovery sequences that burn context for nothing.

Where it works

Context engineering is what turns AI from a toy into a tool. Concrete wins:

Coding assistants that survive a full workday. Claude Code, Devin, Codex — all rely on compaction, notes, and sub-agent fan-out to keep coherent across hundreds of tool calls. Without context engineering, the assistant loses the plot after twenty minutes.
Long-running autonomous tasks. Manus, the general-purpose agent, treats the file system itself as the agent’s extended memory: “unlimited in size, persistent by nature, and directly operable by the agent itself”. Notes and intermediate artifacts go to disk; the model reads them back as needed.
Customer-service and routing agents. The right mix of retrieved knowledge-base snippets, conversation history, and tool definitions turns a chatbot into something that actually resolves tickets. Without curation, it confabulates policy answers from vague training-data priors.
Code review assistants that learn over time. Multi-session memory — pattern libraries that accumulate across PRs — is a context-engineering problem. So is keeping the right subset of repo context loaded for each review.
High-volume production agents where cost matters. A stable system prompt that hits the cache at 95% delivers roughly 10x lower input-token cost than the same prompt with a single rogue timestamp in it. For a startup shipping thousands of agent calls an hour, that 10x is the difference between a margin and a loss.

Where it breaks

Context engineering isn’t magic. Common failure modes:

The right context simply doesn’t exist. No amount of curation fixes missing information. If the model needs a fact that isn’t anywhere in the system, retrieval, or notes, the model will confabulate it. Fix: be honest about the limits of the available context before assuming the model is the problem.
Poisoned memory. Stored notes are read back into context. If untrusted content ever lands in those notes — a webpage summary, a tool result — the agent has effectively injected itself. Fix: isolate memory writes, sanitize on read, treat persistent notes as an attack surface.
Over-aggressive compaction. Summarizing too early loses subtle context whose importance only shows up later. Fix: tune summarization prompts for recall first, then precision. Keep the architecturally significant bits.
Multi-agent fan-out that contradicts itself. Sub-agents without shared context make implicit decisions that conflict with each other. Cognition’s Walden Yan argues the field still hasn’t solved cross-agent context passing; the safe default is single-threaded. Fix: prefer one coherent agent over a fan of confused ones.
Stale retrieval indexes. Pre-loading “always relevant” documents (CLAUDE.md files, legal precedents, financial templates) sounds efficient until the source drifts. Fix: hybrid retrieval — pre-load the high-priority set, lazy-fetch the rest.
KV-cache invalidation by accident. A timestamp in the system prompt, non-deterministic JSON serialization, a tool name that changes order. The cache breaks from that token onward and your bill jumps tenfold. Fix: deterministic serialization, append-only message history, cache-breakpoint markers.

What this article does NOT cover

Implementation patterns, code samples, or framework recommendations (LangGraph, LangSmith, MCP wiring).
Model-by-model context-window sizes, pricing, or capability comparisons.
Prompt engineering as a discipline in its own right — only context engineering, treated as its successor.
The full security model for agent memory and prompt injection; that’s a separate, deep topic.