Skip to content
Search
Techniques8 min read

When to Use a Reasoning Model, and When Thinking Just Costs More

Brian Gagne
When to Use a Reasoning Model, and When Thinking Just Costs More

TL;DR: Use a reasoning model, or turn thinking up, for work with several steps that depend on each other: math, code, debugging, planning and analysis across sources. Keep it low for lookups, classification, short rewrites and anything where speed matters more than depth. The control is an effort setting in each major provider's API. It guides how much the model thinks without capping it, and the only way to know the right level for your task is to test it.

Most AI models now offer some form of thinking: the model works through a problem before it writes the answer you see. That extra work helps on hard problems and wastes time and money on easy ones. This guide covers when to use a reasoning model, when thinking only costs more, and how the settings work, in terms a beginner can act on. If you already know the basics and want tuned starting points per task, our technical guide to reasoning effort goes further.

What a reasoning model does differently

A reasoning model is a large language model trained to work through a problem step by step before it answers. Anthropic's documentation explains why that matters: "A model that answers in a single pass has to get everything right on the first try: no scratch work, no checking, no changing approach halfway through." With thinking on, the model "restates what is being asked, tries approaches, checks intermediate results, and abandons paths that do not hold up." The docs say this is why thinking "improves performance on complex tasks such as math, coding, analysis, and long-running agentic work" (Anthropic, Thinking).

Before reasoning models, people got a similar effect by writing "think step by step" in the prompt, a form of chain-of-thought prompting. Reasoning models do that work on their own, in a separate thinking step.

What thinking costs

Thinking is not free, and the cost is easy to miss because you often don't see the thinking text.

  • Money. Thinking is billed as output. Anthropic says the reasoning tokens "are billed as output tokens, even when the thinking text isn't returned to you" (Anthropic, Thinking). OpenAI says reasoning tokens "still occupy space in the model's context window and are billed as output tokens" (OpenAI, Reasoning models). Google says "Pricing is based on the full thought tokens the model needs to generate" (Google, Thinking).
  • Time. A sample prompt in Anthropic's own guidance puts it simply: "Thinking adds latency and should only be used when it will meaningfully improve answer quality" (Anthropic, Prompting best practices).
  • Room. Thinking counts toward the output limit for the reply (Anthropic, Thinking), so a reply can run out of room before the visible answer is finished. OpenAI recommends "reserving at least 25,000 tokens for reasoning and outputs" when you start experimenting (OpenAI, Reasoning models). Our max tokens entry explains the limit.

Reasoning models also don't always know when to stop. The ThoughtTerminator study found that "reasoning models are poorly calibrated, particularly on easy problems", spending tokens that don't improve accuracy (Pu et al., arXiv 2504.13367). Our article on stripping the wait tokens covers that research in depth.

When to turn thinking up

Thinking pays off when a wrong step early on spoils everything after it, and when you'd want a careful person to check their work. Good candidates:

  • Math, logic and scheduling problems with several constraints.
  • Writing or fixing code, especially bugs that span more than one file.
  • Planning: a project timeline, a migration, a budget with trade-offs.
  • Analysis that has to reconcile several documents or data sources.
  • Agent work, where the model decides what to do next after each tool result.

Google's guidance draws the line in the same place: "Use maximum thinking for advanced coding, math, or multi-step planning" (Google, Thinking).

When thinking just costs more

Keep thinking low, or off where the model allows it, when the task has one obvious answer or when speed matters most:

  • Looking up a fact or answering a simple question.
  • Sorting items into fixed categories, such as tagging support tickets.
  • Pulling fields out of a document into a set format.
  • Short rewrites, summaries and tone changes.
  • Chat where a fast reply matters more than a deep one.

Google's guidance says "Use minimal thinking for fact retrieval or classification" (Google, Thinking). Anthropic describes its lowest effort level as suited to "Simpler tasks that need the best speed and lowest costs, such as subagents" (Anthropic, Effort).

A three-question check

If you're unsure, ask:

  1. Does the task have several steps that depend on each other? If yes, thinking is likely to help.
  2. Would a wrong answer cost more than a slow one? If yes, lean toward more thinking.
  3. Will you run this thousands of times? If yes, test whether a lower setting holds quality, because the cost multiplies.

The settings, by name

On the major APIs, the control is an effort level. Anthropic's API takes output_config.effort, OpenAI's takes reasoning.effort and Google's takes thinking_level. The allowed values and the default vary by model, so check the model's page (Anthropic, Effort; OpenAI, Reasoning models; Google, Thinking). If you use a chat app instead of an API, look for a thinking or reasoning option in the model picker.

Three things about these settings trip people up.

Effort is guidance, and it sets no hard limit. Anthropic calls effort "a behavioral signal, not a strict token budget" (Anthropic, Effort). Older Claude models used a token budget called budget_tokens, and even that was "a target rather than a strict cap"; that option is now deprecated, and Claude 4.7 and later models reject it with a 400 error (Anthropic, Extended thinking).

The hard cap is the output limit. Anthropic's advice for "a hard ceiling on spend" is to use max_tokens: "Effort is soft guidance. max_tokens is a strict limit" (Anthropic, Thinking). Don't set that limit tiny to save money, though. Google advises that to cut cost or latency you "lower thinking_level (low or medium) instead of setting a small max_output_tokens" (Google, Thinking), since a reply that hits the limit can come back cut off.

Some models always think. On some current Claude models, thinking is always on and effort is the main control (Anthropic, Effort). OpenAI notes that supported effort values vary by model (OpenAI, Reasoning models). "Thinking off" isn't always an option, so think of it as low versus high.

How to prompt a reasoning model

One change from older prompting habits is to describe the outcome and let the model choose the steps. OpenAI's reasoning guide says its reasoning-capable GPT-5 models usually work best when you give them "a clear goal, strong constraints" and a clear description of the output, rather than instructions for every intermediate step (OpenAI, Reasoning models). Anthropic's guidance for Claude says "Prefer general instructions over prescriptive steps" and that a prompt such as "think thoroughly" often does better than "a hand-written step-by-step plan" (Anthropic, Prompting best practices).

The Prompt:

Goal: Plan how to move our team's shared files from three old
folders into one new structure without losing access for anyone.

Constraints:
- 14 people, in 3 teams. Each team must keep access to its own files.
- Finance files must stay visible to the finance team only.
- The move has to happen over one weekend.

A good answer is: the new folder structure, the order of steps,
and who needs which access at each step.

Before you answer, check the plan against each constraint and say
in one line how it is met. If a constraint can't be met, say which
one and why instead of guessing.

Why this works: The prompt states the goal, the constraints and what a correct answer looks like, and leaves the method to the model, which is how both vendors recommend prompting reasoning models. The check at the end gives you a quick way to see whether the model kept every constraint. This is a multi-step planning task, so it's a good fit for a higher effort setting.

OpenAI adds a warning worth keeping in mind: "Treat reasoning.effort as a tuning knob, not the primary way to recover quality" (OpenAI, Reasoning models). If answers are poor, fix a vague prompt before you turn effort up.

Test it on your own work

Anthropic's and OpenAI's guides both treat effort as a setting you tune for your own task. Anthropic's best practice is "Test your use case: The impact of effort levels varies by task type" (Anthropic, Effort). Where you start depends on the work. For simple, high-volume tasks such as classification and extraction, start low. For complex or agent work, start at your model's default or the level its guide recommends, which is usually medium or high (Anthropic, Effort; OpenAI, Reasoning models; Google, Thinking). Save the top levels, such as xhigh or max, for tasks where your tests show a gain. OpenAI's advice for its xhigh level is "Only use when your evals show a clear benefit" (OpenAI, Reasoning models).

A simple test: take 10 to 20 real examples of the task, run them at your starting level and one level lower, and compare the answers, the time and the token counts. Keep the lowest level that meets your quality bar, which is also how Google's guide puts it: "pick the lowest thinking_level that meets your quality bar" (Google, Thinking). For a repeatable test set, see our eval harness playbook.

Frequently asked questions

What is a reasoning model?

A reasoning model is a large language model trained to work through a problem before it answers, using extra tokens to think. That thinking helps on complex tasks such as math, coding and analysis. Most major providers offer reasoning models or a reasoning mode with a setting that controls how much it thinks.

When should I use a reasoning model?

Use one for tasks with several dependent steps, such as math, code, debugging, planning and analysis across several sources, and when a wrong answer costs more than a slow one. For lookups, classification and short rewrites, a low setting is usually enough. Test both on your own examples.

Do reasoning models cost more?

Usually, yes. Thinking tokens are billed as output tokens on Anthropic's, OpenAI's and Google's APIs, even when you don't see the thinking text. Higher effort generally means more thinking and a slower reply.

Should I still write "think step by step"?

On a model without a reasoning mode, it can still help, because that's chain-of-thought prompting. On a reasoning model, give the goal, the constraints and what a good answer looks like, and use the effort setting to control depth.

Is higher reasoning effort always better?

No. Gains depend on the task, and reasoning models tend to overthink easy problems. Anthropic's effort docs note that the top setting often costs more for only a small gain, so test your starting level and one level lower on your own examples, and keep the lowest one where quality holds.

Rather have someone set it up for you?

That's a fair call. LTFI, the agentic business operating system from Kief Studio, can set up the work these guides describe and run it with you. No website yet? Kief Studio will build you one for free.

Go further on kief.studio