Context Engineering
A model's context window is a limited budget; curate the smallest set of high-signal tokens it needs for the task.
“Context engineering refers to the set of strategies for curating and maintaining the optimal set of tokens…”
To an executive
AI assistants perform worse when overloaded with information. Deciding carefully what they are shown at each step improves reliability and keeps running costs down.
To an engineer
Treat the context window as a limited budget. Keep tool descriptions distinct and retrieved content relevant, and trim accumulated history so key instructions are not buried.
The idea
Anthropic's applied AI team described context engineering as the next step after prompt engineering. A prompt is only part of what a model sees. Its context also includes system instructions, tool descriptions, retrieved documents and the whole history of the conversation. Context engineering is the work of deciding what goes into that window at each step.
The key observation is that context is a finite resource. As the amount of text grows, models get worse at finding and using what matters — a decline they call context rot. Like a person with limited attention, a model has an attention budget.
Why it works
More is not better. Irrelevant or outdated text competes with the relevant parts for the model's attention.
Agents accumulate. Long tasks pile up tool results and history until the important instructions are buried.
Where it applies
- Designing agents and assistants that run for many steps.
- Retrieval systems, where it is tempting to stuff in every document that might help.
- Tool design. Each tool description costs context, and overlapping tools confuse the model.
How to apply
- Aim for the smallest high-signal context. Clear instructions at the right level of detail, a few representative examples, and only the data this step needs.
- Retrieve just in time. Keep lightweight references, such as file paths or IDs, and let the model load details through tools when needed.
- Compact long histories. Summarise and restart when the context nears its limit.
- Take notes outside the window. Let agents write progress to memory or files and read back what they need.
- Split work across sub-agents that return condensed results to a lead agent.
Example scenario
An internal coding assistant loaded the whole repository's documentation into every request and became slower and less accurate as the docs grew. The team changed it to load only a short project overview up front and to search files on demand.
Answers improved, costs fell, and adding more documentation no longer made the assistant worse.
Questions to ask
- What does the model actually need to see for this step, and what is noise?
- What can be loaded on demand instead of up front?
- How do we keep long-running tasks from filling the context with stale history?
When not to use it
- Short, single-step tasks where everything relevant comfortably fits and curation adds little.
How it misleads you
Over-trimming starves the model. Cutting context to save tokens can remove exactly the detail the model needed. Measure quality on an evaluation set as you curate, not just token counts.
In the wild
Further reading
Last reviewed · Suggest an improvement