Temperature · Sampling Settings
Temperature (Sampling)
What is temperature?
- Controls the randomness and creativity of answers
- Low for consistency, high for variety
- Set it for the task
Changing temperature does not change what the model knows. What it changes first is how widely answers diverge when you ask the same question ten times. Near 0, nine out of ten runs give almost the same sentence, making review and reproduction easy. Above 1, sentence structure and word choice change every time, so getting the same result again becomes hard. In practice, temperature is not a knob for making the model smarter; it is a knob that trades reproducibility against variety. Knowledge and reasoning ability stay the same, and only the range of variation is adjusted. For an A/B test, record the value and fix it on both sides, or the comparison will not hold.
Value ranges differ by model. OpenAI models use 0–2, many open-source models use 0–1, and the default is usually around 0.7. In practice, teams typically use 0–0.3 for classification, extraction, summarization, and code; 0.4–0.7 for customer-support replies or report drafts; and 0.9–1.2 for naming, slogans, and concept brainstorming. It is easy to confuse with Top-p, but temperature flattens or sharpens the entire probability distribution, while Top-p keeps only the top candidates by cumulative probability and cuts the rest. Pushing both hard at once makes their effects overlap and hard to control, so you normally fix one and adjust the other. Note also that standard chat screens do not expose this value; you can change it only through the API or a playground.
The most common misconception is that setting temperature to 0 stops the model from saying wrong things. Temperature is not a fact-checking mechanism but a control for variation in wording, so even at 0 the model can state nonexistent clause numbers or court cases in a confident tone. Worse, with no variation it is wrong the same way every time, so reviewers may fail to notice the error. Conversely, using the default 0.7 for tasks that need a single correct answer, such as internal policy guidance or quote calculation, can lead to incidents where the same question yields a refund deadline of 7 days one time and 14 days the next. Even at temperature 0, fully identical output is not guaranteed because of the order of computation, so policy guidance should be handled by attaching the source text, not by adjusting temperature. When you need creative output, too, changing the conditions and examples in the prompt fails less often than raising temperature first.
John KimAuthor of “B2B Marketing Design” and CEO of Nanum Consulting
Hyerang ParkContent storytelling and AI-driven brand content specialistDerived from a statistical concept that controls the 'sharpness' of a probability distribution (softmax temperature), it is a standard setting for controlling output diversity in generative AI.
Consider a hypothetical case: the CS team at Company A, a home appliance maker with 240 employees and annual revenue in the KRW 60 billion (≈ US$44,890,000) range. They connected a support chatbot via API and ran it for three months with the temperature left at the default 0.7. Asked the same 'exchange deadline' question, it answered 7 days one time and 14 days the next, and 31 complaints about wrong information were filed in two months.
- When Team Lead Park on the CS team pulled 50 identical questions from support logs and compared them, the refund-policy answers had split into four different versions. Someone in the meeting said, 'This looks like a settings problem, not a people problem.'
- Kim, a manager on the development team, lowered the temperature to 0.2 for policy-guidance calls only, and rewrote the prompt so that the relevant paragraph of the internal policy is pasted in verbatim as the basis for each answer.
- They re-ran the same 50 questions to check reproducibility. Answers with mismatched policy figures dropped from 11 to 1, and Team Lead Park approved the setting for production.
- For gift promotion copy, by contrast, they raised the temperature to 0.9 and collected 20 options a day, and Lee on the marketing team adopted 3 of them as actual banner copy.
- Six weeks later, complaints about incorrect exchange and refund guidance fell from 15 a month to 2, and the average time agents spent on second-pass re-checks dropped from 9 minutes to 4 per case.