What it is
Temperature is a setting that controls how much randomness a language model uses when it picks its next word. Turn it down and the model plays safe, choosing the most likely words and giving much the same answer each time. Turn it up and the model takes more chances, choosing less likely words and giving answers that vary more, sometimes interestingly and sometimes badly.
To see why a setting like this exists at all, you need one idea from how a language model writes. It produces text one small piece at a time. Those pieces are called tokens: often a whole short word, sometimes part of a word or a punctuation mark. At each step the model doesn't decide on one next token. It produces a list of every token it knows, each with a probability. After "The cat sat on the", the list might give "mat" a 40 percent chance, "floor" 20 percent, "sofa" 10 percent, and thousands of other tokens tiny slivers of the rest.
Something then has to choose one token from that list. That choosing step is called sampling, and it happens hundreds of times in every answer, during what is called inference, the moment a trained model is actually used.
The easiest way to picture sampling is a weighted dice. Imagine a dice with thousands of faces, one for every token, where each face is sized according to its probability. "Mat" gets a huge face, "floor" a large one, "volcano" a face so thin it almost never comes up. The model rolls the dice, writes down whichever token lands face up, and then builds a new dice for the next step.
Temperature changes the shape of the dice before it's rolled.
- Low temperature makes the big faces bigger and the small faces smaller. The favourite, "mat", now takes up nearly the whole dice, and the roll comes out the same almost every time. At the extreme, often written as temperature 0, the model skips the dice entirely and always takes the single most likely token. That shortcut is called greedy decoding.
- Temperature 1 leaves the dice as the model made it. The probabilities are used exactly as the model calculated them.
- High temperature flattens the dice. The big faces shrink and the small ones grow, so "sofa", "windowsill" and even odder choices come up more often. Push it far enough and every face becomes nearly the same size, and the text dissolves into nonsense.
Temperature doesn't make the model know more, think harder, or become more or less accurate on its own. It only changes how boldly the model picks from options it has already calculated. That's worth holding onto, because a lot of confusion about temperature comes from treating it as a creativity dial or a truth dial. It's a randomness dial.
The name comes from physics. In the 1860s the physicist Ludwig Boltzmann described how particles in a gas spread themselves across different energy levels. At low temperatures they crowd into the lowest-energy states; at high temperatures they spread out more evenly. The formula he worked out, now called the Boltzmann distribution, has the same shape as the calculation a language model uses to turn its raw scores into probabilities, a step called softmax. Computer scientists borrowed the formula and kept the word. So when you lower a model's temperature, you are, mathematically, cooling the dice.
There's a close cousin you'll meet in the same settings panel, called top-p (or nucleus sampling). Instead of reshaping the whole dice, top-p throws away the long tail of unlikely faces first. With top-p set to 0.9, the model keeps only the most likely tokens that together add up to 90 percent of the probability, and rolls among those. A related setting, top-k, keeps a fixed number of the most likely tokens, say the top 40. Both are guard rails: they let you have some variety without ever landing on a token that was a one-in-a-million shot. OpenAI's API documentation, for one, recommends changing temperature or top-p but not both, because they pull on the same thing.
Where can you actually change it? Mostly not in the chat apps. At the time of writing, the everyday apps for ChatGPT, Claude and Gemini don't show a temperature control; the company picks a setting for you. You get the dial in developer tools, often called playgrounds, such as Google AI Studio, and in the programming interfaces behind them, plus some third-party writing and chat tools built on top. Features change often, so check the current version of whatever tool you use.
Why it matters
It explains why the same question gets different answers. Ask a chatbot the same thing in two fresh chats and you'll usually get two different replies. That isn't the model changing its mind or being broken. It's the dice being rolled again. Knowing this stops you from treating any single reply as "the" answer the AI gives, and it's why regenerating a response is a perfectly sensible thing to do when the first one is weak.
It's a trade between reliability and variety. For tasks with one right answer, such as pulling a date out of an invoice, classifying a support ticket or producing text in a strict format (see structured output), randomness is only a source of errors, so low temperature is the traditional choice. For tasks where you want options, such as brainstorming names, taglines or story ideas, a higher temperature gives you a wider spread to choose from. Neither is better in general. The right setting depends on whether you want the same answer twice.
Always picking the top word writes badly. It sounds as if the safest setting should give the best text. For longer writing it usually doesn't. Always taking the single likeliest word tends to produce flat, repetitive prose that can loop on the same phrase. Stephen Wolfram made this point in his widely read February 2023 essay on how ChatGPT works: always choosing the top-ranked word tends to give a flat essay, while sometimes choosing lower-ranked words at random gives better results, and he noted that a temperature of 0.8 seemed to work well for essays, with no real theory behind the number. Some randomness is part of what makes a model's writing sound human.
It changes how you test prompts. If you're comparing two versions of a prompt, randomness can fool you. One run of each tells you very little, because the difference you see may just be a different roll. Running each version several times, or at low temperature where you're allowed to, gives a much fairer comparison. That's the heart of good evals.
The newest models are taking the dial away. At the time of writing, several of the big AI companies either restrict temperature on their reasoning models (models that work through a problem step by step before answering) or advise you to leave it alone. That is covered in the mistakes section below, but it matters here too: many of the old rules of thumb, such as "use temperature 0 for anything factual", don't carry over to the newest models, and the vendor's current documentation is the only reliable guide.
Real-world examples
From a physics formula to "softening" a model (1868 and 2015)
Temperature entered AI through physics. Ludwig Boltzmann formulated his distribution in 1868 while studying gases in thermal equilibrium: it gives the probability that a system is in a given state, depending on that state's energy and on the temperature. Low temperature concentrates probability on the lowest-energy states; high temperature spreads it out. The softmax function that machine learning uses to turn a model's raw scores into probabilities has exactly this form, which is why the dividing number in it came to be called temperature.
One of the best-known uses of the idea before chatbots came in March 2015, in a paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean called "Distilling the Knowledge in a Neural Network". They wanted to train a small, cheap model to copy a large one. Their trick was to raise the temperature of the large model's softmax so that, instead of giving one answer with near-total confidence, it produced a "softer" spread of probabilities, showing which wrong answers it thought were nearly right. The small model learned from that softer spread at the same high temperature, then went back to temperature 1 for normal use. The technique, called distillation, is still how many small, fast models are made today. It is a good illustration that temperature doesn't add knowledge: it only reveals, or hides, how the model's confidence is spread out.
The unicorn paper: why the likeliest words make bad text (2019)
In April 2019, Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes and Yejin Choi posted "The Curious Case of Neural Text Degeneration", later published at the ICLR 2020 conference. They asked why language models that were good at predicting text were so often bad at writing it.
Their opening figure used the famous prompt about scientists discovering English-speaking unicorns in the Andes, fed to GPT-2 Large. A method that hunts for the single most probable continuation (beam search, a more thorough relative of always taking the top word) produced text that fell into degenerate repetition. Pure random sampling from the full, untrimmed list went the other way and produced incoherent gibberish, because the long tail of unlikely tokens kept getting picked. The authors also showed that once a phrase repeats, the model grows more confident that it will repeat again, a feedback loop that traps high-probability decoding.
Their fix was top-p, which they called nucleus sampling: sample only from the smallest group of tokens that together make up most of the probability, and let that group grow or shrink with the model's confidence. They noted that turning temperature down improves quality but reduces diversity, and that temperature alone doesn't suppress the unreliable tail well enough. Top-p became a standard setting, and it's the reason many playgrounds show two sampling controls rather than one.
Temperature 0, and 80 different answers (September 2025)
Developers have long assumed that temperature 0 makes a model deterministic: same input, same output, every time. On 10 September 2025, Thinking Machines Lab, the AI company founded by former OpenAI chief technology officer Mira Murati, published a post by Horace He and colleagues called "Defeating Nondeterminism in LLM Inference" that tested the assumption directly.
They sent the same prompt 1,000 times at temperature 0 to Qwen3-235B, a large open model, running on a standard serving setup. They got 80 different completions. The most common appeared 78 times, and the answers stayed identical for the first 102 tokens before splitting at token 103. The popular explanation is that computer chips add numbers in slightly different orders, and tiny rounding differences snowball. The post argues that the main culprit is something more mundane: a busy server groups your request with other people's requests, the size of that group changes with traffic, and the calculations come out very slightly differently depending on the group size. When the team rewrote the key calculations so they no longer depended on the group size, all 1,000 completions came out identical.
The lesson for everyone else is simple. Temperature 0 removes the deliberate dice roll, but it doesn't guarantee the same answer, because the hardware underneath can still wobble. Anthropic's current migration guide makes the same point in one line, noting that temperature 0 "never guaranteed identical outputs" on its earlier models.
Is temperature the creativity dial? (2024)
It's common advice that you turn temperature up for creative work. In May 2024, Max Peeperkorn, Tom Kouwenhoven, Dan Brown and Anna Jordanous tested that claim in a paper presented at the International Conference on Computational Creativity, titled "Is Temperature the Creativity Parameter of Large Language Models?".
They gave Llama 2-Chat 70B, an open model from Meta, the prompt "Write a story." and generated stories at seven temperatures from 0.001 to 2.0. They measured how varied the stories were, and asked human participants to rate them for novelty, typicality, cohesion and coherence. Temperature had only a weak relationship with novelty and a moderate relationship with incoherence: hotter stories were slightly more original and noticeably more likely to stop making sense. There was no relationship with cohesion or typicality. At the lowest setting the model could only produce one other unique story besides its default one. The authors concluded that the effect of temperature on creativity is far weaker and more nuanced than the "creativity parameter" label suggests.
For a working creator, that matches experience. Turning up the heat gives you more variety, not more imagination. If you want genuinely different ideas, changing the prompt, for example by asking for ten options in different styles, does more than any slider.
An everyday case: the bakery's product descriptions
Imagine Tom, who runs a small online bakery and uses an AI writing tool that happens to have a temperature slider. He has two jobs for it.
The first is turning his supplier's ingredient lists into the standard allergen line for each product. He sets temperature low, near the tool's minimum, because he wants the same wording every time and no creative flourishes. Even so, he reads every line before it goes on the website, because low temperature makes the output more consistent, not more correct.
The second is writing the short, warm description for each new seasonal loaf. At the low setting, every description opens with "Indulge in" and uses the word "artisanal". He raises the temperature to around the middle of the range and asks for five versions at a time. Now he gets real variety, and he picks the best and edits it. When he pushed the slider to the top once, two of the five came back with odd, half-invented phrases, so he settled on the middle.
Try it yourself
You need a web browser. Step 1 needs no account. Step 2 uses Google AI Studio, which at the time of writing is free with an ordinary Google account; check its current terms and limits yourself. Allow about thirty minutes.
- See the dice change shape. The Language Model article walks you through the free Transformer Explainer from Georgia Tech, which runs a small real model in your browser with a temperature slider. Go back to it with one new goal: type a sentence, set temperature low and note how much of the probability sits on the top word, then set it high and watch the bars even out. The tool also has top-k and top-p options; switch to top-p and see how the list of candidate words shrinks.
- Try a real model with the dial. Open Google AI Studio (aistudio.google.com) and sign in with your Google account. Start a new chat and find the run settings panel, where temperature appears as a slider. Paste this prompt:
Suggest one name for a small bakery that specialises in sourdough. Reply with the name only.
- Run it five times at a low temperature. Set temperature to about 0.2 and run the prompt five times, starting fresh or regenerating each time. Write down the names. You'll probably see the same name, or close variations, again and again.
- Run it five times at a high temperature. Set temperature to about 1.5 and repeat. Write down the five names. Expect more variety, and perhaps one or two odd ones.
- Notice the model's advice. At the time of writing, Google recommends leaving its Gemini 3 models at the default temperature of 1.0 and warns that lower values can cause looping or weaker reasoning on complex tasks. If you're using one of those models, that's a good reminder that the "right" setting depends on the model, not just the task.
- Get variety without a slider. Most chat apps don't show temperature, but you can ask for variety in words. In any chatbot, try this:
Give me 10 names for a sourdough bakery. Make each one different in style: some plain, some playful, some old-fashioned, one that is a pun. Number them.
Compare the spread with what you got at high temperature. Asking for range in the prompt is usually more useful than randomness alone.
- Get consistency without a slider. For tasks where you want the same shape of answer every time, spell out the format and give an example, as in this prompt:
Rewrite this ingredient list as a single allergen line. Use exactly this format: "Contains: [allergens]. May contain: [traces]." Do not add anything else.
Ingredients: [paste the list]
Run it three times. The wording should barely move, because the prompt has left very little for the dice to decide.
Common mistakes and misunderstandings
Thinking temperature 0 is fully deterministic. It removes the deliberate randomness, but as the Thinking Machines experiment showed, busy servers can still produce different answers from identical requests. If you truly need the same output every time, store the answer rather than regenerating it.
Thinking low temperature makes answers true. A low temperature makes the model give its most likely answer consistently. If its most likely answer is wrong, you'll get the same wrong answer every time, delivered with confidence. It doesn't prevent hallucination. Checking facts, giving the model your source documents, and asking it to say when it isn't sure do far more.
Thinking high temperature makes the model more creative. It makes the model more random, which gives you more variety and more risk of nonsense. The 2024 study found only a weak link to originality. Better prompts, more options, and your own editing are what produce genuinely creative results.
Turning temperature and top-p up together. Both loosen the model's choices. Pushing both at once compounds the effect and makes results harder to control. Change one at a time, and leave the other at its default.
Assuming every model accepts the setting. At the time of writing, Anthropic's documentation says its newer Claude models reject any non-default temperature, top-p or top-k value with an error, and recommends guiding the model through the prompt instead. Microsoft's documentation for OpenAI's reasoning models served through Azure lists temperature and top-p among the parameters most of them don't support. Google recommends keeping Gemini 3 at its default. These rules change with each model generation, so check the current documentation before relying on an old setting.
Copying magic numbers. Advice like "0.7 for writing, 0 for code" circulates widely. Treat any number as a starting point, not a rule. The same number behaves differently on different models, and there's no universal scale. Test with your own task, several runs at each setting.
Judging a prompt from one reply. Because every reply is a roll of the dice, one great or terrible answer tells you little. Run it a few times before deciding a prompt works, or doesn't. The technique of generating several answers and taking the most common one, called self-consistency, turns this randomness into a strength.
Blaming the model for variation you asked for. If a tool lets you set a high temperature and the answers wander, that's the setting working. Before concluding the model is unreliable, check what the setting is.
Related elements
- Language Model: the next-word prediction loop that temperature adjusts, with the Transformer Explainer walkthrough.
- Tokens: the small pieces of text the model rolls its weighted dice over.
- Inference: the moment a model runs, when sampling and temperature actually happen.
- Self-consistency: sampling several answers on purpose and taking the most common one.
- Structured Output: fixed formats where low randomness and a clear template keep answers consistent.
- Hallucination: why a lower temperature makes wrong answers consistent rather than rare.
- Extended Thinking: the reasoning mode on many newer models, which often fixes or removes the temperature setting.
- Evals: testing prompts across many runs so random variation doesn't fool you.
- Iterative Refinement: regenerating and improving answers, which works because each roll is different.
- Prompt: the other main lever on output, and often a better one than any slider.