Temperature
Temperature is a scalar that scales logits before softmax. Values below 1 sharpen the distribution (more deterministic); values above 1 flatten it (more random). Temperature 0 is equivalent to greedy decoding.
What is Temperature?
Temperature is a scalar that scales logits before softmax. Values below 1 sharpen the distribution (more deterministic); values above 1 flatten it (more random). Temperature 0 is equivalent to greedy decoding.
Temperature is a scalar that scales logits before softmax. Values below 1 sharpen the distribution (more deterministic); values above 1 flatten it (more random). Temperature 0 is equivalent to greedy decoding.
Where is it used?
Every production LLM API (OpenAI, Anthropic, HuggingFace Inference) exposes a temperature parameter. Code generation uses low temperature (~0.2) for precision; creative writing uses higher (~0.7-1.0). GPT-4 defaults to 1.0.
How to build it
Divide logits by temperature before softmax: `probs = F.softmax(logits / temperature, dim=-1)`. Use `temperature = 0.7` as a sensible default for balanced chat. Guard against division by zero.
Code
A practical example:
Further reading
Want to go deeper? These resources cover Temperature in more detail: