Loss
A loss function measures the difference between a model's prediction and the true target, producing a scalar to minimize. It defines what the model is optimizing toward.
What is Loss?
A loss function measures the difference between a model's prediction and the true target, producing a scalar to minimize. It defines what the model is optimizing toward.
A loss function measures the difference between a model's prediction and the true target, producing a scalar to minimize. It defines what the model is optimizing toward.
Where is it used?
LLMs use cross-entropy loss over the vocabulary; regression tasks use MSE; contrastive learning uses triplet or InfoNCE loss for embedding models.
How to build it
Use `torch.nn.CrossEntropyLoss()` with logits and integer labels, or `F.mse_loss` for regression, and observe how `.backward()` populates gradients for every parameter.