Probability
Probability quantifies how likely an event is, ranging from 0 to 1. In ML, probability distributions describe model outputs — an LLM produces a probability over the vocabulary at each step.
What is Probability?
Probability quantifies how likely an event is, ranging from 0 to 1. In ML, probability distributions describe model outputs — an LLM produces a probability over the vocabulary at each step.
Probability quantifies how likely an event is, ranging from 0 to 1. In ML, probability distributions describe model outputs — an LLM produces a probability over the vocabulary at each step.
Where is it used?
LLMs output a categorical distribution over tokens via softmax; temperature sampling in GPT and Claude scales these probabilities before drawing the next token.
How to build it
Use `torch.softmax(logits, dim=-1)` to convert logits to probabilities, verify they sum to 1 with `.sum()`, and visualize with `matplotlib.pyplot.bar`.