Most guides tell you temperature controls creativity. Turn it up for creative writing, turn it down for factual tasks. Simple slider, simple mental model.

That mental model is wrong. And it's costing you output quality in both directions.

What Temperature Actually Does

Temperature is a distribution sharpness parameter. It doesn't generate ideas. It doesn't make the model "think harder." It rescales the probability distribution over the model's vocabulary before a token gets selected.

At temperature 0.2, the model's top prediction dominates. If "the" has a 40% probability and "a" has 15%, that gap widens. The model almost always picks "the."

At temperature 1.5, that gap shrinks. "The" might drop to 25% and "a" rises to 18%. Now several tokens have a real shot at being selected. You get more variety in word choice, but the model isn't being more creative. It's being more random.

Randomness and creativity are not the same thing.

Peeperkorn et al. (2024) tested this directly. They measured four dimensions of creativity in narrative generation: novelty, typicality, cohesion, and coherence. Temperature had a weak correlation with novelty and a moderate correlation with incoherence. It had zero relationship with cohesion or typicality. Their conclusion: "randomness alone does not capture a complex phenomenon such as creativity."

Put differently, cranking temperature gives you stranger word choices and less coherent structure. That's not creativity. That's noise.

The Flat Middle Range

Here's what surprised the researchers: temperature changes between 0.0 and 1.0 often don't matter at all.

Renze et al. (2024) tested nine LLMs across five prompting techniques, sweeping temperature from 0.0 to 1.6. Changes from 0.0 to 1.0 had no statistically significant impact on problem-solving performance. None. Across models, across techniques, across problem domains.

They also found that increasing temperature up to 1.0 does not create hallucinations that lead to incorrect answers. The "set temperature to 0 so it doesn't hallucinate" advice? Not supported by their data.

A separate clinical study confirmed this from the other direction. Hallucination rates were 65.9% at default temperature and 66.5% at temperature zero. Statistically identical. What actually reduced hallucinations to 44.2% was better prompt design.

The lever that works is your prompt. Temperature is a rounding error by comparison.

Temperature Zero Isn't Even Deterministic

If you're setting temperature to 0 expecting identical outputs every time, you're getting fooled.

Thinking Machines Lab ran 1,000 completions of the same prompt at temperature 0 with Qwen3-235B. They got 80 unique completions. The most common one appeared only 78 times out of 1,000. Tokens matched perfectly for the first 102 positions, then diverged at token 103.

The cause wasn't floating-point rounding. It was batch invariance failure. Inference servers dynamically adjust batch sizes based on current load. Different batch sizes produce different softmax calculations, and at temperature 0 where multiple tokens have nearly identical probabilities, that tiny numerical difference picks a different winner.

Their fix achieves perfect reproducibility but runs roughly 60% slower. No major API provider has implemented it.

So when someone tells you "just set temperature to 0 for consistent results," know that you're getting mostly-consistent results at best. And you're giving up the variance that could actually help you detect when the model is uncertain.

The Coupling Problem Nobody Mentions

Temperature and top-p are not independent controls. Adjusting one silently changes what the other does.

Top-p (nucleus sampling) keeps tokens from the top of the probability distribution until their cumulative probability reaches your threshold. If you set top-p to 0.9, the model considers tokens that together account for 90% of the probability mass.

But temperature changes the shape of that distribution before top-p applies its cutoff. Higher temperature flattens the distribution, which means more tokens fall within that 90% window. Lower temperature sharpens it, so fewer tokens qualify.

Set temperature to 1.5 and top-p to 0.9, and you might have 50 candidate tokens. Set temperature to 0.3 with the same top-p of 0.9, and you might have 5. You didn't change top-p, but the effective diversity of your candidate pool changed dramatically.

This is why advice like "use temperature for creativity and top-p for diversity" misleads people. They're coupled parameters. Changing one redefines the other.

Min-P: The Parameter You Should Know About

The open-source community has largely moved past the temperature-plus-top-p combination. The replacement is min-p, and it was accepted as an oral presentation at ICLR 2025, ranked 18th among all submissions.

Min-p sets a dynamic floor: only consider tokens whose probability is at least some fraction of the top token's probability. If the top token has 60% probability and you set min-p to 0.1, only tokens with at least 6% probability survive.

The key difference from top-p: min-p adapts to the model's confidence. When the model is very sure (one token dominates), min-p naturally restricts the pool. When the model is uncertain (flat distribution), min-p lets more candidates through. It's responsive instead of fixed.

Human evaluators preferred min-p outputs over top-p, epsilon-sampling, and Mirostat for both quality and creativity. Min-p at 0.1 with temperature 1.5 outperformed all alternatives in the benchmarks.

It's available in llama.cpp, vLLM, HuggingFace Transformers, and Ollama. It is not available in the OpenAI, Anthropic, or Google APIs. If you run models locally, you have access to better sampling than commercial API users.

Model Size Changes the Rules

Li et al. (June 2025) tested models across three size tiers: 1B-4B parameters, 6B-13B, and 40B-80B.

Small models performed more creatively at lower temperatures. Large models maintained stable performance from temperature 0 all the way to 2 with no sudden degradation.

This means temperature advice calibrated to GPT-4 or Claude doesn't transfer to a 7B model you're running locally. The "right" temperature depends on which model you're using. Generic advice is generic for a reason: it doesn't account for this.

The Industry Is Taking the Knobs Away

OpenAI's o1, o3, and GPT-5 reasoning models have temperature fixed at 1.0. You cannot change it. Anthropic restricts temperature to 0.0-1.0, half the range of OpenAI's standard models. With Claude 4.5, you can't use temperature and top-p together at all. AWS Bedrock defaults reasoning orchestration to temperature 0 with top-k and top-p unset.

The pattern is clear: the more capable the model, the less the provider lets you touch sampling parameters. These models handle their own token selection internally, and the providers have decided users adjusting temperature does more harm than good.

What to Actually Do

Here's a prompt that demonstrates why prompt design matters more than temperature. Try this at temperature 0.3 and temperature 1.2. You'll get structurally similar, useful results at both settings:

You are a technical writer creating documentation for a REST API endpoint.

Write the documentation for a POST /users endpoint that:
- Accepts JSON with fields: name (string, required), email (string, required), role (enum: admin, member, viewer)
- Returns 201 with the created user object on success
- Returns 422 with field-level validation errors

Use this structure: Description, Request Body, Response (success), Response (error), Example curl command.

Be precise. No filler sentences.

This prompt works because it constrains the output through structure, not through temperature. The model knows what format to produce, what fields to include, and what tone to use. Temperature becomes almost irrelevant when your prompt is this specific.

POST /users

Creates a new user account.

Request Body json { "name": "Jane Park", "email": "jane@example.com", "role": "member" }

201 Created json { "id": "usr_8x2kf9", "name": "Jane Park", "email": "jane@example.com", "role": "member", "created_at": "2026-03-28T14:30:00Z" }

422 Unprocessable Entity json { "errors": [ {"field": "email", "message": "must be a valid email address"}, {"field": "role", "message": "must be one of: admin, member, viewer"} ] }

The specificity of the prompt did the work. Not the temperature setting.

Practical defaults if you need a starting point:

For commercial APIs (OpenAI, Anthropic, Google): keep temperature between 0.3 and 0.8. Leave top-p at the provider default. Don't touch both at once.

For local models with min-p support: use temperature 0.8-1.2 with min-p 0.05-0.1. This combination gives you better diversity control than anything available through commercial APIs.

For code generation: 0.2-0.4 temperature works well. For creative text, stay in the 0.7-1.0 range. The research consistently shows that moderate settings outperform extremes.

And stop expecting temperature zero to solve your accuracy problems. If the model is hallucinating, the fix is in your prompt, your context, or your retrieval pipeline. Not in a sampling parameter.

Build the Skill That Actually Matters

Temperature is one number. Your prompt is hundreds of words of precise instruction. Which one do you think has more influence over output quality?

If your team is spending time tweaking sampling parameters instead of learning to write better prompts, the ROI is backwards. Kief Studio runs hands-on prompt engineering training where we work through exactly these kinds of misconceptions with real examples and live iteration. Connect with us on Discord or book a session at kief.studio/contact.

sampling-parameters

Temperature Is Not a Creativity Dial: What Sampling Parameters Actually Do and When to Change Them