Your agent isn't failing because the model is dumb. It's failing because by turn 30, the context window is full of stale tool outputs, redundant observations, and a system prompt buried under 80K tokens of noise.

This is the context engineering problem. And it's now the single biggest bottleneck in agentic AI systems.

Prompt Engineering Got You Here. Context Engineering Gets You There.

Prompt engineering asks: "How should I phrase this?" Context engineering asks: "What information does this model need access to right now, at this specific turn, to make the right decision?"

When your agent runs for 3 turns, the distinction barely matters. When it runs for 50 turns across multiple tool calls, file reads, and sub-agent handoffs, context engineering is the entire game.

Gartner declared context engineering "in" and prompt engineering "out" as of mid-2025. They predict context engineering features will be embedded in 80% of AI development tools by 2028. The reason is simple: 57% of organizations now have agents in production, but 32% cite output quality as their top barrier -- and the failures trace to context management, not model capability (LangChain State of Agent Engineering, 2025).

Context Rot Is Real and It Hits Early

Here's the part most teams miss: bigger context windows don't fix this.

Chroma Research tested 18 frontier models in July 2025, including GPT-4.1, Claude Opus 4, and Gemini 2.5. Eleven of twelve models dropped below 50% of their baseline performance at just 32K tokens. GPT-4o fell from 99.3% to 69.7% accuracy (Chroma Research).

At 100K tokens, the transformer computes 10 billion pairwise attention relationships. Every token you add dilutes attention on the tokens that actually matter. The discipline isn't "fit more in." It's "keep less, keep it sharp."

The Token Budget Template

Here's a working allocation for a 200K-token agent window, synthesized from Anthropic's engineering docs, the LangChain agent report, and JetBrains' research:

Slice	Budget	What Goes Here
System prompt + ground truth	5-10%	Persona, rules, constraints, factual grounding
Tool definitions	5-10%	Active tool schemas only (rotate dynamically for 30+)
Retrieved context (RAG)	15-20%	Fresh retrieval per turn, ranked by relevance
Conversation history (verbatim)	10-15%	Last 6-10 turns, original user intent
Compressed history	10-15%	Older turns summarized: decisions, outcomes, open issues
Working memory / scratchpad	5-10%	Agent's current plan, progress, active goals
Generation headroom	20-30%	Output tokens + reasoning

The critical rule: never let RAG pre-loading exceed 60% of your budget before the user's actual request enters the window. If your retrieval eats most of the context before the model even sees the task, you've already lost.

Mask Before You Summarize

The instinct when context fills up is to summarize old turns. JetBrains Research found this actually makes agents worse (NeurIPS 2025, arXiv:2508.21433).

LLM summarization "smooths over" failure signals. When an agent hits an error on turn 12 and the summary compresses it into "attempted approach X," the agent loses the specific signal that should tell it to stop trying that path. It keeps going, burning tokens without progress. JetBrains calls this "trajectory elongation."

Their fix is simple: observation masking. Replace old tool outputs with a placeholder like [output from turn 12 -- masked]. The agent still knows the action happened and can request the full output if needed, but the stale data doesn't pollute attention.

With Qwen3-Coder 480B, masking achieved a 54.8% solve rate at $0.61 per instance. Summarization got roughly the same solve rate at $0.64 per instance. Masking is cheaper and doesn't destroy failure signals.

The rule: mask by default. Summarize only when the context is critically full and the agent needs the semantic content of old turns to proceed.

Context Poisoning: The 100-Hour Bug

Google's Gemini 2.5 Pro played Pokemon for 813 hours. During one run, the model confused Full Heal (cures status conditions) with Full Restore (heals HP). That single hallucination entered the agent's goal scratchpad.

Result: a 100-hour loop. The agent kept losing to the Elite Four and going back to train, because it "believed" it had a healing item it didn't have. The error reinforced itself every time the agent re-read its own scratchpad (Drew Breunig).

The DeepMind team identified this as "context poisoning" in the Gemini 2.5 technical report. Once an error enters the context, the model treats it as ground truth. There's no self-correction mechanism.

The defense: treat scratchpad content as mutable, not append-only. Periodically re-validate working memory against external state. If your agent maintains a plan, re-derive it from actual observations every N turns instead of trusting the accumulated narrative.

Building a Context Manager: The Prompt

Here's a system prompt pattern for an agent that manages its own context window across long runs.

The Prompt:

You are an autonomous agent with a limited context window. You MUST actively manage what stays in your working memory.

After every 5 tool calls, perform a CONTEXT AUDIT:
1. List each item currently in your scratchpad/working memory
2. For each item, classify as: ACTIVE (needed for current sub-task), REFERENCE (might need later), or STALE (task completed or superseded)
3. Remove all STALE items immediately
4. For REFERENCE items older than 10 turns, compress to one-line summaries
5. Keep ACTIVE items verbatim -- never summarize in-progress work

CONTEXT RULES:
- The user's original request stays verbatim until task completion. Never summarize it.
- Tool outputs from completed sub-tasks get replaced with: [MASKED: {tool_name} returned {one-line result summary}]
- If you notice your plan contradicts observed tool output, trust the tool output and revise the plan.
- Before each action, state which scratchpad items informed your decision.

Current scratchpad:
{scratchpad_contents}

Original user request:
{original_request}

Why This Works: The prompt forces the agent to periodically audit its own context rather than letting it accumulate unchecked. The classification system (ACTIVE/REFERENCE/STALE) gives the model a concrete framework for deciding what stays and what goes. The rule about trusting tool output over plan narrative directly defends against context poisoning.

Expected Output (after a context audit):

Context Audit (Turn 15): - "Deploy to staging environment" -- ACTIVE, current sub-task - "Database migration completed successfully" -- STALE, removing (sub-task done, result: 3 tables migrated) - "User wants zero-downtime deployment with rollback capability" -- ACTIVE, original intent, keeping verbatim - "npm build output: 847 modules, 0 errors" -- STALE, masking: [MASKED: npm build returned success, 847 modules, 0 errors] - "Rollback script location: /deploy/rollback.sh" -- REFERENCE, keeping (needed if deployment fails)

Proceeding with staging deployment. Decision informed by: original user request (zero-downtime requirement) and rollback script reference.

Three Rules That Cover 80% of Context Failures

1. Keep the original user intent verbatim, always. It's your anchor. Everything the agent does should trace back to what was actually asked. Summarize everything else before you touch this.

2. Mask tool outputs from completed sub-tasks. Don't summarize them, don't keep them, don't let them compete for attention with your current work. A one-line placeholder is enough.

3. Re-validate working memory against reality every N turns. Don't trust the narrative the agent has built about the world. Check actual file contents, actual API responses, actual state. This is your defense against context poisoning.

The Mercor APEX-Agents benchmark tested 8 frontier agents on 480 real professional tasks. Best score: 24% completion. The failures came from "losing context, failing to locate the right document, and misreading fragmented workplace signals" (Mercor, Jan 2026). These are context engineering problems, not intelligence problems.

The models are capable. The question is whether you're giving them the right information at the right time.

If your team is building agentic systems and hitting the context wall, Kief Studio runs hands-on training sessions on context engineering, prompt architecture, and agent orchestration. Connect with us on Discord or schedule a session.

LLM context window

Context Engineering for Agents: Manage What Your LLM Sees Across 50 Turns Without Losing the Thread

Context Engineering Beats Prompt Engineering: How to Design Token Budgets That Actually Work