ModelRefs / Temperature — AI Glossary

Temperature — AI Glossary

A decoding parameter that controls randomness in token sampling — 0 is deterministic, higher values produce more variation.

Overview

At temperature 0 the model picks the highest-probability token every step (greedy decoding). At 1.0 it samples proportionally to the softmax distribution. Use low temperature for extraction/factual tasks and higher temperature for creative generation.

Reference details

Topic	inference
Last reviewed	2026-06-24

Example: Zero is not quite deterministic

Temperature 0 selects the highest-probability token each step, which is as close to reproducible as an API gets — but batching, hardware and floating-point non-associativity can still shift a tie. Treat 0 as strongly repeatable, not as a guarantee, and pin a seed where the provider offers one.

Commonly confused with

Temperature is not a creativity dial. It rescales the probability distribution: higher values raise the chance of low-probability tokens, which reads as creative when it works and as incoherent when it does not. It does not add ideas, only variance.

When to use it

Reach for it when:

Low for extraction, classification and anything with one right answer
Higher for drafting, brainstorming or generating varied synthetic data
Higher when you sample several outputs and vote across them

Reach for something else when:

Raising it to fix dull output — that is usually a prompt problem
Non-zero while debugging, where reproducibility matters more
Tuning it before the prompt and the schema are settled

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Temperature — AI Glossary.

Frequently asked questions

What is Temperature?