What is a Temperature (AI)?

Under the hood, a language model produces a probability distribution over its entire vocabulary for the next token. Temperature rescales that distribution before sampling: dividing the logits by a value below 1 sharpens the distribution toward the most likely tokens, producing more repeatable and conservative output, while a value above 1 flattens it, making less-likely tokens more competitive and output more varied. A temperature of 0 effectively becomes greedy decoding, always picking the top token. Choosing temperature is a tradeoff between reliability and creativity. Low temperature suits tasks with a single correct answer, such as code generation, structured data extraction, or factual Q&A, where consistency matters more than variety. Higher temperature suits brainstorming or creative writing, where some randomness produces more useful variation across runs.