Skip to content
Search
DevelopmentGlossary

Max Tokens (Output Length Limit)

Every request has two budgets: the context window, which limits input and output together, and the output cap you set with a max tokens parameter. Providers name it differently. Anthropic's Messages API calls it max_tokens, and OpenAI's Responses API calls it max_output_tokens. Each model also has its own ceiling. On reasoning models the thinking usually counts toward the same cap: OpenAI's reference says max_output_tokens includes reasoning tokens, and Anthropic's thinking docs say thinking tokens count toward max_tokens alongside the response text. A low cap can leave no room for the answer.

The API tells you when the cap cut a response short through a stop or finish reason, so check it and never treat a truncated answer as complete. A truncated JSON object won't parse, which is a common source of structured output errors. Set the cap from the longest valid answer you expect, plus a margin. If you want shorter answers, ask for brevity in the prompt as well, because the cap only cuts text off. It's one of the sampling-adjacent settings alongside temperature and top-p.

Example

# Set a cap, then check whether the answer was cut off
response = client.responses.create(
    model=MODEL,
    max_output_tokens=800,   # longest valid answer plus a margin
    input=prompt,
)
if response.status == "incomplete":
    # The cap was reached. Raise the cap, shorten the task,
    # or ask the model to continue. Don't use the partial output as final.
    handle_truncation(response)

When to use it

  • Controlling cost and latency on high-volume calls
  • Protecting a pipeline from runaway output
  • Sizing room for reasoning plus the final answer

When not to use it

  • As your only way to get short answers; say what length you want in the prompt
  • Setting it so low that valid answers, or a reasoning model's thinking, get cut off