What is LLM temperature in simple terms?

It is a setting that decides how much randomness goes into picking each next token. Mechanically, the model's raw scores are divided by the temperature before they are turned into probabilities, so a low value sharpens the choice toward the top candidate and a high value flattens it.

Does temperature 0 always give the same answer?

No. It makes the model take the top-scored token, but Anthropic's documentation states that results will not be fully deterministic even at 0.0, and OpenAI describes determinism with its seed option as best effort and not guaranteed.

Is a higher temperature more creative?

It is more varied, which is not the same thing. A higher value gives lower-ranked words a larger share of the probability. Providers suggest higher values for creative tasks, but the model does not gain ideas it did not already score.

Does lowering the temperature stop hallucinations?

No. Lowering it makes the model repeat its top choice more reliably, and if that choice is false it will be false each time. Temperature changes how the model picks, not whether its scores point at something true.

What is the difference between temperature and top-p?

Temperature rescales all the scores before the draw. Top-p removes the unlikely tail and draws from what is left. OpenAI's API reference recommends altering one or the other but not both.

LLM · SAMPLING

What Is LLM Temperature? Why the Same Prompt Gives Different Answers

LLM temperature is a setting that controls how much a language model's choice of next token varies. The model's raw scores are divided by the temperature before they become probabilities, so a low value lets the top candidate dominate and a high value gives lower-ranked candidates a real chance. It changes how the model picks, not what it knows.

One-pager titled "What is LLM temperature?" with the subtitle "The same scores, reshaped before the pick". One panel of four dimmed raw-score bars, labelled food, toy, bone and ball, feeds a horizontal temperature slider with three knobs, one above each column, each with an arrow down into probability bars for the same four candidates. Column one is divided by 0.5, labelled sharper: food 99 percent, toy 1 percent, bone and ball under 1 percent. Column two is divided by 1, labelled same: 85, 8, 4 and 3 percent. Column three is divided by 2, labelled flatter: 58, 18, 13 and 11 percent. A REMEMBER band notes: it reshapes the scores, low is steadier but not fixed (three example runs read food, food, toy), it adds no knowledge. — One-pager: one set of raw scores divided by a temperature, giving sharper, unchanged or flatter probabilities. Values are illustrative.

What temperature does to the scores

At each step, a language model produces one raw score for every token in its vocabulary. A function called softmax turns those scores into probabilities that add up to 1. The next-token prediction page walks through that step.

Temperature sits just before the softmax. Every raw score is divided by a number T. Hinton, Vinyals and Dean wrote the formula down in 2015 for classifiers: the probability of option i is exp(z_i / T) divided by the sum of exp(z_j / T) over all options, where z are the raw scores, the logits. T is "normally set to 1", and a higher T "produces a softer probability distribution". Text models use the same softmax. Hugging Face's documentation describes temperature as the value used to modulate the next token probabilities, with 1.0 as the fallback when a model's generation config does not set one.

In plain terms, a T below 1 widens the gaps between scores, so the leading token takes a larger share. A T above 1 shrinks the gaps, so the share spreads out. At T = 1 nothing changes. One thing never moves: the order. Dividing every score by the same positive number leaves the best candidate best.

The model has read this and must pick the next word

The dog chased the cat because it stole his ___

Temperature1.0

Always the top wordMore surprising

85%

food

toy

bone

ball

About 85 in 100 runs pick “food”.

Press the button to run the same prompt 10 times.

Illustrative numbers. A real model scores every token in its vocabulary. At 1 the scores are left alone.

Why the same prompt gives different answers

Most chat generation does not take the top token every time. It samples. Hugging Face's guide describes sampling as randomly selecting a token based on the probability distribution over the model's whole vocabulary, so every token with a non-zero probability has a chance. Greedy decoding, by contrast, takes the most likely token at each step.

Temperature controls how lopsided that draw is. Anthropic's API documentation describes its temperature parameter as the amount of randomness injected into the response. With a sharp distribution, most runs land on the same word. With a flat one, runs spread out.

The differences then compound. Each chosen token is fed back in as input for the next step, as the generation loop page explains. One different word early on changes everything the model reads afterward, so two runs can drift far apart even though they began from the same prompt and the same scores.

Temperature 0 and why runs can still differ

As T shrinks toward 0, the top token takes almost all of the probability. In the limit that is greedy decoding. Greedy output has its own problem. Hugging Face notes that greedy search breaks down on longer outputs because it begins to repeat itself, and Holtzman and colleagues describe likelihood-maximizing decoding as producing text that is bland and strangely repetitive.

A setting of 0 also does not promise identical output. Anthropic's documentation says that even with a temperature of 0.0, the results will not be fully deterministic. OpenAI's reference describes its seed option, labelled deprecated and in beta, as a best effort to sample deterministically, and says determinism is not guaranteed. Neither page gives a mechanism, so this page does not guess at one.

The safe reading is that low temperature makes repeats more likely. It is not a guarantee, and an application that needs exactly reproducible text should not rely on the setting alone.

What temperature does not do

It does not add knowledge. Temperature rescales scores the model has already produced, so a fact the model never learned does not become likely at any setting. It also does not make a model creative the way a person is. Providers suggest lower values for analytical tasks and higher ones for creative tasks, and what the setting actually does is let lower-ranked words through. Those words can read as fresher or as worse. Holtzman and colleagues found that decoding choices alone can change text quality a great deal, even from exactly the same model.

It does not fix hallucination either. At a low setting the model keeps taking its top choice, and if that choice leads to a false statement, the statement repeats. The page on what an AI hallucination is covers what does help.

Temperature is also not the only sampling control. Top-p sampling cuts off the unlikely tail instead of rescaling the scores. OpenAI's API reference recommends changing temperature or top_p but not both.

Ranges and defaults depend on the provider

There is no universal scale. As of 2026-10-11, Anthropic's Messages API documents temperature as a number from 0.0 to 1.0 with a default of 1.0, and marks the parameter deprecated: models released after Claude Opus 4.6 do not support setting it, accept a value of 1.0 for backward compatibility, and reject other values with a 400 error. OpenAI's API reference documents a range of 0 to 2.

Some models restrict it. OpenAI's migration guide for its GPT-6 family says to remove temperature and top_p from a request when reasoning effort is not none. Reasoning models are a separate design, covered on the reasoning model page.

So a number copied from one tutorial may mean something different elsewhere, or be rejected. For any real use, the current documentation for the specific model is the source to check.

FAQ

What is LLM temperature in simple terms?: It is a setting that decides how much randomness goes into picking each next token. Mechanically, the model's raw scores are divided by the temperature before they are turned into probabilities, so a low value sharpens the choice toward the top candidate and a high value flattens it.
Does temperature 0 always give the same answer?: No. It makes the model take the top-scored token, but Anthropic's documentation states that results will not be fully deterministic even at 0.0, and OpenAI describes determinism with its seed option as best effort and not guaranteed.
Is a higher temperature more creative?: It is more varied, which is not the same thing. A higher value gives lower-ranked words a larger share of the probability. Providers suggest higher values for creative tasks, but the model does not gain ideas it did not already score.
Does lowering the temperature stop hallucinations?: No. Lowering it makes the model repeat its top choice more reliably, and if that choice is false it will be false each time. Temperature changes how the model picks, not whether its scores point at something true.
What is the difference between temperature and top-p?: Temperature rescales all the scores before the draw. Top-p removes the unlikely tail and draws from what is left. OpenAI's API reference recommends altering one or the other but not both.

Sources

Last updated 2026-10-11