Skip to content
Search
DevelopmentGlossary

Temperature

Temperature scales the logit distribution before sampling the next token. At temperature=0, the model always picks the most likely next token (near-deterministic). At temperature=1, it samples proportionally to probability. Above 1, unlikely tokens get relatively boosted, producing more varied and surprising output.

Production defaults: classification / extraction / structured output → 0.0-0.2. General chat → 0.5-0.7. Creative writing → 0.7-1.0. Above 1.0 is usually noise. Temperature=0 is not fully deterministic on frontier models (there's other stochasticity); rely on `seed` parameters when available for reproducibility. Providers differ here, and some have restricted or removed sampling parameters on newer models and API versions. Anthropic's API versioning notes, for example, say its current API version rejects temperature, top_p and top_k on every model. Check the API reference for the model and API version you use before you rely on this setting.

Example

# Choosing temperature by task, on an API that still accepts the parameter

response = client.chat.completions.create(
    model=MODEL,
    messages=[...],
    temperature=0.0  # classification -> low
)

response = client.chat.completions.create(
    model=MODEL,
    messages=[...],
    temperature=0.7  # draft marketing copy -> medium
)

When to use it

  • You want reproducibility: use 0.0
  • You want variety across multiple samples: use 0.5-0.8
  • Creative brainstorming: up to 1.0

When not to use it

  • Setting temperature > 1 without a specific reason (quality cliff)
  • Assuming temperature=0 = fully deterministic (not on most modern APIs)