Skip to content
Search
Techniques11 min read

Context Engineering vs Prompt Engineering: Design Token Budgets

Brian Gagne
Context Engineering vs Prompt Engineering: Design Token Budgets

TL;DR: Context engineering vs prompt engineering comes down to scope. Prompt engineering is how you word the instruction. Context engineering is everything the model sees when it answers: instructions, examples, retrieved documents, tool definitions, history and state. Past a point, rewording stops helping, and the fix is what you load, where it sits and what you leave out.

Context engineering vs prompt engineering is a question you hit the day a rewrite stops working. You've rewritten the system prompt six times, added emphasis and bolted on "make sure to include X", and the output still misses key facts. Usually the cause is what the model saw before it started writing.

Anthropic's engineering team defines prompt engineering as "methods for writing and organizing LLM instructions for optimal outcomes" and context engineering as "curating and maintaining the optimal set of tokens (information) during LLM inference." It describes context as "a finite resource with diminishing marginal returns" (Anthropic, Effective context engineering for AI agents).

Context engineering vs prompt engineering: the key differences

The table sets the two side by side. The definitions come from Anthropic's post above; the rest follows from them.

Prompt engineering Context engineering
Scope The instruction you write Everything in the window: instructions, examples, documents, tools, history, state
When it happens Mostly once, when you write the prompt Every turn: "the curation phase happens each time we decide what to pass to the model"
Single or multi-turn Shines on single requests Matters most in long chats and agents
Typical failure The model misreads what you want The model has the wrong material, too much of it, or the key fact buried
Typical fix Reword, add an example, set the format Retrieve less, reorder, summarize, split into sub-agents
Tools and memory Out of scope In scope: which tools load, what's saved outside the window

Is one a subset of the other?

Anthropic's answer is direct: "At Anthropic, we view context engineering as the natural progression of prompt engineering" (Anthropic). The instruction is still part of the context, so writing it well still matters. It just stops being the only lever once a system pulls in documents, tools and history.

Why more tokens can make answers worse

Anthropic explains that models have an "attention budget" that they draw on when parsing large volumes of context (Anthropic). The research backs it up.

  • Position matters. In "Lost in the Middle", performance was often highest when the relevant information sat at the start or end of the input, and dropped when the model had to use information in the middle, even for long-context models (Liu et al., arXiv 2307.03172; our digest).
  • Length alone hurts. Across five models, performance "still degrades substantially" as input length increases, by 13.9% to 85%, even with perfect retrieval, and still fell when the irrelevant text was swapped for whitespace (Du et al., arXiv 2510.05381).
  • Focused beats full. Chroma's study of 18 models compared focused prompts of about 300 tokens with full ones of about 113,000 tokens and reported "significantly higher performance on focused prompts compared to full prompts" (Chroma, Context Rot).

That decline is called context rot, and our guide to context rot in long chats covers when to start fresh. The practical rule: treat an advertised context window size as a ceiling and budget well under it.

Black and white photo of hands moving three chosen sheets from a tall stack of papers into a single folder
Choosing the few pages that matter for the next step is a large part of building a good context.

Budget the context window

Treat the window as a fixed token budget: a token spent on one thing can't be spent on another. These ranges are a starting point for a 128k window; measure your own system and adjust.

Slice Starting budget Notes
System instructions 2k to 5k, fixed Role, rules, constraints. At 20k, you're describing in words what examples should show.
Current state and tool outputs 20k to 40k, changes every turn What the model must act on now.
Retrieved documents 10k to 30k Filter hard. Fewer, relevant paragraphs usually beat many loose ones.
Conversation history 5k to 15k, sliding Summarize older turns.
Headroom 10% to 15% For a large tool result or a retrieval spike.

Also leave room for the response. When you hit the ceiling, something has to go, and deciding what goes is the heart of context engineering.

A context window split into five bands: instructions, current state, retrieved documents, history and headroom, with token ranges
Context engineering vs prompt engineering: the prompt is one band of the window, and the budget covers all of it.

For the system prompt band, Anthropic advises writing at "the right altitude", which it calls "the Goldilocks zone between two common failure modes": rules so rigid they break, and guidance so vague it gives the model nothing to act on (Anthropic). Our guide on how to write a system prompt shows what that looks like in practice.

Four operations: write, select, compress, isolate

Context work falls into four moves.

  • Write: save information outside the window and load it back when needed.
  • Select: load only what this step needs. Retrieval (RAG) is selection, and so is tool filtering. RAG-MCP retrieved only the relevant tool descriptions and "more than triples tool selection accuracy (43.13% vs 13.62% baseline)" (Gan and Sun, arXiv 2505.03275). Anthropic reported its tool search approach gave "an 85% reduction in token usage while maintaining access to your full tool library" (Anthropic, Advanced tool use).
  • Compress: cut tokens and keep the signal. Claude Code uses auto-compaction, "which summarizes conversation history when approaching context limits," and /compact with instructions "tells Claude what to preserve during summarization" (Claude Code docs, Manage costs).
  • Isolate: give sub-agents their own clean windows and have them return short summaries. See agent handoff protocols for what to pass between them.

Research points the same way. Agentic Context Engineering treats contexts "as evolving playbooks that accumulate, refine, and organize strategies," and "matches the top-ranked production-level agent on the overall average" of the AppWorld leaderboard while using a smaller open model (Zhang et al., arXiv 2510.04618).

Just-in-time retrieval vs loading everything up front

Instead of loading every document at the start, agents built for just-in-time retrieval "maintain lightweight identifiers (file paths, stored queries, web links, etc.)" and pull the data in with tools when they need it. Anthropic describes Claude Code as a hybrid: it loads its project instruction files up front and uses search tools to fetch other files as it goes (Anthropic).

Compaction, notes and sub-agents for long tasks

Anthropic names three techniques for work that outlasts one window (Anthropic):

  • Compaction "is the practice of taking a conversation nearing the context window limit, summarizing its contents" and starting a new window with the summary. Anthropic calls clearing old tool results "one of the safest lightest touch forms of compaction."
  • Structured note-taking, "or agentic memory, is a technique where the agent regularly writes notes" outside the window, such as a to-do list or a NOTES.md file, and reads them back later.
  • Sub-agents explore in their own windows and return "a condensed, distilled summary of its work," often 1,000 to 2,000 tokens. Our guide to sub-agent delegation prompts covers when to spawn one.

Two worked failures

Here is context engineering vs prompt engineering in two short examples. Both are illustrations to show the difference in fixes, and you can reproduce them on your own material.

Side-by-side comparison of a failure fixed by rewording the request and a failure fixed by changing the loaded documents
The first failure is fixed in the wording; the second only changes when the loaded documents change.

Failure 1: rewording fixes it

You paste a 12-message customer thread and ask: "Summarize this for the team." You get a long, even-handed recap when you wanted the decision points. The context was fine; the request was ambiguous. A prompt-engineering fix works:

Summarize this thread for our support lead in 3 bullets:
1. The customer's problem, in one sentence.
2. What we have already promised them, with dates.
3. The next step and who owns it.
If any of these isn't in the thread, write "not stated".

Amelia's guide to writing AI prompts that give you useful answers covers this kind of fix in depth.

Failure 2: only a context change fixes it

A support assistant gets the whole 200-page product manual and the question "How do I reset my API key?" The right section is in there, but it sits in the middle of a very long input, which is where the Lost in the Middle results say models use information least well. Adding "make sure to use the key management section" asks the model to find a needle you could have handed it. The context fix is to load only that section, put it first, and say what to ignore.

The Prompt:

You are a technical support agent for Acme API Platform.

## What you're looking at
The user is asking about API key management. Below is the matching
section of our docs (last updated 2026-05-01). Use only this section.
If the answer isn't here, say so.

## Documentation
[API Key Management section only, about 2 pages]

## What to ignore
Don't bring up billing or account deletion. The user didn't ask.

## Format
Numbered steps, at most 5. Include the dashboard URL for the action.

User question: How do I reset my API key?

Why this works: The model no longer has to work out which of 200 pages matters. The context is one section, placed first, with a rule for what to skip and a fixed answer shape, so the instruction can stay short.

Context engineering vs prompt engineering in an agent

Agents show the gap clearly. Picture an agent on step 40 of a migration that forgets a decision it made at step 5, because 35 tool outputs have piled up since. "Remember what you decided in step 5" is a prompt fix and rarely holds. The context fixes are the long-task techniques above: write decisions to a notes file and reload them each turn, clear old tool results, and send big searches to a sub-agent that returns a short summary. Our guide to context engineering for agents walks through a 50-turn example.

Which one to fix first

A quick test tells you whether a failure is a context engineering vs prompt engineering problem. You can run it in a chat window in a few minutes.

  1. Hand it the perfect context. Rerun the failing request with only the material a person would need, placed first. If the answer is now right, the problem was what the model saw.
  2. If it still fails, fix the wording. With the right material in front of it, a wrong answer points at the request: make the task, the audience and the format explicit, as in Failure 1.
  3. If it works early and fails late, watch the growth. A request that works on turn 1 and slips on turn 30 is the context rot pattern from the research above. Compact the history, keep notes outside the window, or start a fresh session with a short summary.

You will often need both. The test tells you which one to reach for first.

Where harness engineering fits

Harness engineering is a newer term you'll see next to these two. A 2026 paper describes an agent harness as "the external execution system around a model that organizes a task run" (Pan et al., arXiv 2603.25723). In that framing, the harness runs the loop, the context is what the loop feeds the model each turn, and the prompt is one part of that context.

Context engineering vs prompt engineering examples: the usual fixes

Problem The prompt-engineering reflex The context-engineering fix
Output ignores key facts "Make sure to include X, Y and Z." Put X, Y and Z near the start of the context.
Tone drifts "Write in a professional tone." Add three short examples of the tone you want.
Invented details "Only use verified information." Supply the source documents and cut the irrelevant ones.
Agent forgets earlier steps "Remember what you did in step 2." Write state to a notes file and reload it each turn.
Costs too high Shorten the prompt. Trim history, cache the stable prefix, route simple requests to a smaller model.

On cost, prompt caching bills repeated prefix tokens at a reduced rate on Anthropic's API, which rewards keeping instructions and examples stable at the top (Anthropic, Prompt caching). Our prompt caching guide covers how to structure for it.

Start here

One quick way to see the difference in your own work is to measure one system. What share of the window is fixed instructions and what share is live content? Where does the critical information sit? What gets dropped when the window fills? If instructions take most of the space, trim them, expand the live content and put the important material first. More techniques are in the Techniques pillar.

Frequently asked questions

What is context engineering in AI?

Context engineering is choosing and arranging everything a model sees when it answers: instructions, examples, retrieved documents, tool definitions, history and saved state. Anthropic defines it as curating and maintaining the best set of tokens during inference, and the choice is made again every turn.

Context engineering vs prompt engineering: what's the main difference?

Prompt engineering is how you word the instruction. Context engineering covers the whole window and decides what gets in, where it goes and what's left out. When rewording stops helping, the fix is usually in the context.

Is context engineering better than prompt engineering?

Neither replaces the other. In context engineering vs prompt engineering, the prompt is one part of the context, so a clear instruction still matters. Context work matters more as a system adds documents, tools and long history.

What is replacing prompt engineering?

Nothing is replacing it outright. Anthropic calls context engineering "the natural progression of prompt engineering," and harness engineering adds the loop that runs an agent around both.

Is prompt engineering outdated?

No. Clear instructions are still part of every good context. They stop being the main lever once what the model sees shapes the answer more than how the request is worded. Brian makes the same case in Prompt Engineering Is Infrastructure, Not a Party Trick.

Why does a bigger context window not always help?

Models get less reliable as input grows. In one study, performance fell 13.9% to 85% with longer inputs even when every relevant fact was retrievable, and information in the middle of a long input is used less well than information at the start or end.

Context engineering vs prompt engineering vs harness engineering: how do they fit?

The harness is the system that runs the loop around the model. Context is what that loop feeds the model each turn, and the prompt is one part of that context. Fix the wording first when the material is right, and the context when it isn't.

What is a token budget?

It's a planned split of the context window between instructions, current state, retrieved documents, history and headroom, so the most important content always fits and the least important is dropped first.

Rather have someone set it up for you?

That's a fair call. LTFI, the agentic business operating system from Kief Studio, can set up the work these guides describe and run it with you. No website yet? Kief Studio will build you one for free.

Go further on kief.studio