Skip to content
Search
TechniquesGlossary

Context Rot

The term spread in mid-2025 and was popularized and tested in a July 2025 technical report by Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance. The researchers tested 18 models on tasks where they held difficulty constant and changed only the input length, and found that performance became less reliable as the input grew. It builds on earlier work such as Lost in the Middle (2023), which found that models used information at the start and end of a long context better than information in the middle.

The practical lesson is to fill the context window with what the task needs and leave the rest out, however large the window is. Anthropic's guide to context engineering for AI agents describes the same effect and recommends keeping context small and relevant. Long agent sessions are where it shows up first, as tool output, old turns and stale plans pile up. The usual fixes are retrieving only what each step needs (RAG), compaction, handing side work to a subagent with its own clean context, and better chunking. Qurtoo's article on context engineering for agents across 50 turns walks through a full setup.

Example

Before each step of this task, rebuild your working context:
1. The task goal, in one paragraph.
2. Decisions made so far, as a bulleted list of no more than 10 items.
3. Only the files or documents this step needs.
Once you've recorded what you learned from a tool result, drop the raw output. If you need it again, fetch it again.

When to use it

  • Long conversations and agent sessions that run for many turns
  • Prompts that include large documents or many retrieved passages
  • Answers get worse when you add more material, even relevant material

When not to use it

  • Short single-turn prompts, where the effect is small
  • Blaming context length when the answer was never in the context at all