🧠 What Is Context Engineering for AI Agents?
🧠 Learn what context engineering is, how it differs from prompt engineering, and the 4 core operations (Write, Select, Compress, Isolate) that keep AI agents reliable.
🔑 Key Takeaways
- 🎯 Context engineering is the discipline of designing everything that enters an LLM's context window on every call: instructions, tools, retrieved knowledge, memory, conversation state, and constraints.
- ✍️ Prompt engineering optimizes the instruction. Context engineering manages the full, dynamic information environment across multi-step runs.
- 📏 The context window is finite and degrades with length and noise. Every token competes for attention.
- 🔄 In practice it reduces to four operations: Write ✍️ | Select 🎯 | Compress 🗜️ | Isolate 🧱
🌅 Introduction
Early LLM work was mostly about finding the right words. Agent work is different. 🤖 An agent generates new data every turn: tool outputs, observations, partial plans. That data must be stored, refined, or discarded before the next call.
The question changes from:
❌ "How do I phrase this?"
✅ "What configuration of information is most likely to produce the right behavior on this turn?"
That broader practice is context engineering.
⚔️ 1. Context Engineering vs Prompt Engineering
| 📌 Dimension | ✍️ Prompt Engineering | 🧠 Context Engineering |
|---|---|---|
| 🔭 Scope | Instructions / one message | Entire context window over time |
| 💾 State | Mostly stateless | Stateful, evolves across steps |
| 🛠️ Core skill | Phrasing and examples | Information-flow architecture |
| 💥 Failure mode | Vague or misread instruction | Context rot, distraction, contradictory state |
| 🎬 Typical use | Single completion or chat turn | Multi-step, long-running agents |
Prompt engineering is still necessary: clear instructions, good examples, and format constraints matter. But it is a subset of context engineering, not a substitute. Once an agent runs for many turns, failures shift from "the instruction was ambiguous" to "the model saw the wrong, stale, or overwhelming set of tokens."
💡 Prompt engineering shapes the ask. Context engineering shapes what the model knows before it answers.
🧩 2. Anatomy of the Context Window
A typical agent call assembles:
- 📜 System / developer instructions
- 🔧 Tool definitions and schemas
- 📚 Retrieved knowledge (RAG / hybrid search)
- 🧠 Durable memory (preferences, facts, policies, past episodes)
- 💬 Recent conversation (working memory)
- 📦 Tool results from the current run
- 🚧 Constraints, output format, safety rules
Two realities follow:
- 📏 The window is finite. Every token competes for attention.
- 🎯 Position and noise matter. Models show "lost-in-the-middle" effects: a long, low-signal transcript hurts quality even when the right fact is somewhere inside it.
⚠️ More context is not automatically better. High-signal, well-scoped, timely context is better.
🤖 3. Why Agents Force This Discipline
A single-turn task can survive a well-written prompt. An agent can't, because every loop iteration produces new observations, and each one might matter next turn, next session, or never.
The harness must decide:
- ✅ What to keep
- 💾 What to store outside the window
- 🔍 What to retrieve later
- 🗑️ What to drop
Context assembly is the first step of every loop iteration. Context engineering makes that step deliberate.
🛠️ 4. The Four Core Operations
✍️ Write: persist outside the window
- 🗂️ Use memory stores, scratchpads, logs, and files
- 🎯 Information stops competing for attention every turn
- ⚠️ Without a write path, the only memory is the current window, and it disappears when the run ends
🎯 Select: pull in only the relevant slice
- 🔍 Vector or hybrid retrieval, scoped memory search
- 🧹 Filtered tool results, ranking by relevance and recency
- ⚠️ Noise and cross-user leakage are selection failures
🗜️ Compress: keep the window high-signal
- 📝 Thread summaries, context cards, progressive disclosure of older turns
- ✂️ Trimmed tool output
- ⚠️ Unbounded appending leads to context rot and rising token cost
🧱 Isolate: give subtasks clean boundaries
- 🤝 Sub-agents that return only a summary
- 🪟 Separate windows for research and synthesis
- 🔐 Memory scoped per user or tenant
- ⚠️ One contaminated window can derail an otherwise correct plan
🔗 5. How Retrieval, Memory, and Compression Work Together
- 🔍 Retrieval supplies external knowledge on demand
- 🧠 Memory supplies continuity across sessions
- 🗜️ Compression keeps the active window usable
1
2
3
4
5
6
7
8
9
10
11
12
13
User task
│
▼
[Select] ← memory store + retriever (scoped by user/tenant)
│
▼
[Compress] ← summarize old history, trim tool output
│
▼
[Assemble context] → LLM reasons → tool call
│ │
│ ▼
└──────── [Write] ◄──────── tool result / new facts
📋 6. Practical Principles
- ✅ Prefer high-signal tokens over volume
- 🔄 Rebuild context every turn instead of blindly appending
- 🔐 Scope memory and retrieval (user / tenant / agent) before ranking
- 📦 Treat tool results as first-class context that must be managed, not just appended
- 📊 Measure it: context quality shows up in task success, token cost, and long-run consistency
🚫 7. Common Anti-Patterns
- 📚 Append-only history: adding everything from every turn. Context rot and token cost both grow.
- 🗑️ Dumping raw tool output: passing a 5,000-token JSON when only 3 fields mattered.
- 🧰 Too many tools: large tool lists confuse the model and increase wrong-tool calls.
- 🕰️ Stale memory: outdated preferences or facts create contradictory state.
- 🔓 No scoping: one user's memory leaking into another user's context.
💻 8. A Minimal Example
1
2
3
4
5
6
def build_context(task, user_id, history):
memory = memory_store.search(task, scope=user_id, k=3) # select
docs = retriever.search(task, k=4) # select
older, recent = history[:-6], history[-6:]
summary = summarize(older) # compress
return [system_prompt, *tool_schemas, *memory, *docs, summary, *recent, task]
📊 9. How to Evaluate Context Quality
- 📈 Task success vs context size: does more context actually help?
- 💰 Token cost per completed task
- 🔁 Long-run consistency: at turn 30, does the agent still follow the instructions from turn 1?
- 🧪 Ablation tests: remove one context component and measure how much the output changes
❓ FAQ
🤔 Is prompt engineering dead?
No. It remains essential for instructions, format, and examples. It is a subset of context engineering.
No. It remains essential for instructions, format, and examples. It is a subset of context engineering.
🪟 Context window vs memory?
The context window is what the model sees on the current call (working memory). Durable memory is stored state that can be selected into future windows.
The context window is what the model sees on the current call (working memory). Durable memory is stored state that can be selected into future windows.
🔍 Do I still need RAG?
Often yes. RAG is one of the main "select" mechanisms. Context engineering decides when and how to use it.
Often yes. RAG is one of the main "select" mechanisms. Context engineering decides when and how to use it.
🗜️ Where do summaries and context cards fit?
They are compression techniques that turn long history or large tool results into a compact, prompt-ready package.
They are compression techniques that turn long history or large tool results into a compact, prompt-ready package.
🥀 What is context rot?
The gradual drop in answer quality as the window fills with low-signal, stale, or contradictory tokens.
The gradual drop in answer quality as the window fills with low-signal, stale, or contradictory tokens.
🔄 How does this relate to the agent loop and harness?
Context assembly is the first step of each loop iteration. The harness is the code that performs write, select, compress, and isolate around the model call.
Context assembly is the first step of each loop iteration. The harness is the code that performs write, select, compress, and isolate around the model call.
🎉 Final Thought
🧠 Better agents don't come from better prompts alone. They come from better information flow. Design what goes into the window, and the model's behavior follows.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article