Prompt Caching
Many requests share a long, fixed beginning: a system prompt, tool definitions, a reference document, a set of examples. With caching, the provider keeps the work it did on that prefix for a short time, and the next request that begins with exactly the same tokens reuses it. Cached input is billed at a discount. OpenAI's prompt caching guide says caching is on by default for supported models and that reuse requires the entire prefix to match. Anthropic's prompt caching docs let you mark where the cacheable prefix ends, with a five-minute default lifetime and a longer option at extra cost.
To benefit, put stable content first and anything that changes (the user's question, today's date, retrieved results) last. One changed character early in the prompt breaks the match for everything after it. Caching affects what you pay and how long you wait, while the model reads the same prompt either way. Qurtoo's article on prompt caching and reuse works through prompt layouts in detail.
Example
# Order a prompt so the cache can hit
1. Tool definitions (same on every call)
2. System prompt and rules (same on every call)
3. Reference documents (same for this customer)
4. Few-shot examples (same on every call)
--- everything above is the cacheable prefix ---
5. Conversation so far (grows each turn)
6. Today's date and the new question (changes every call)When to use it
- Long system prompts or documents reused across many calls
- Multi-turn chats and agent loops that resend the same history
- High-volume pipelines where input tokens are most of the bill
When not to use it
- Short prompts below the provider's minimum cacheable length
- Calls spaced further apart than the cache lifetime
- Prompts that put changing content (timestamps, user names) at the top
Articles that use this
- Prompt Caching and Reuse: Design Prompts That Actually Hit the Cache at ScaleTechniques
- Reasoning Effort: How to Set Thinking Depth Instead of Writing "Think Step by Step"Techniques
- How to Write a System Prompt: Four Patterns From the Prompts AI Vendors PublishTechniques
- Context Engineering vs Prompt Engineering: How to Design Token Budgets That Actually WorkTechniques
- Prompt Decomposition: Break One Complex Prompt Into Five Simple Ones and Get Better Results for Less MoneyAgentic

