Temperature
Temperature scales the logit distribution before sampling the next token. At temperature=0, the model always picks the most likely next token (near-deterministic). At temperature=1, it samples proportionally to probability. Above 1, unlikely tokens get relatively boosted, producing more varied and surprising output.
Production defaults: classification / extraction / structured output → 0.0-0.2. General chat → 0.5-0.7. Creative writing → 0.7-1.0. Above 1.0 is usually noise. Temperature=0 is not fully deterministic on frontier models (there's other stochasticity); rely on `seed` parameters when available for reproducibility. Providers differ here, and some have restricted or removed sampling parameters on newer models and API versions. Anthropic's API versioning notes, for example, say its current API version rejects temperature, top_p and top_k on every model. Check the API reference for the model and API version you use before you rely on this setting.
Example
# Choosing temperature by task, on an API that still accepts the parameter
response = client.chat.completions.create(
model=MODEL,
messages=[...],
temperature=0.0 # classification -> low
)
response = client.chat.completions.create(
model=MODEL,
messages=[...],
temperature=0.7 # draft marketing copy -> medium
)When to use it
- You want reproducibility: use 0.0
- You want variety across multiple samples: use 0.5-0.8
- Creative brainstorming: up to 1.0
When not to use it
- Setting temperature > 1 without a specific reason (quality cliff)
- Assuming temperature=0 = fully deterministic (not on most modern APIs)
Articles that use this
- The System Prompt Architecture for Coding Agents: What Claude Code and Cursor Get RightDevelopment
- XML Tags Are the New Markdown: Why Closed Delimiters Beat Free-Form PromptsTechniques
- Your Agent's Planner Is Fine. Its Aggregator Is What's BrokenAgentic
- Delete Your Temperature Parameter: Why Self-Consistency Sampling Is Dead on Reasoning Models Like Gemini 3.5 FlashTechniques
- Legal Document Review Prompting: Clause Extraction Without Hallucinated CitationsApplied

