Training Loss
Training loss is the average cross-entropy (or other) loss computed on the training set during training, measuring how well the model fits the data it is learning from.
What is Training Loss?
Training loss is the average cross-entropy (or other) loss computed on the training set during training, measuring how well the model fits the data it is learning from.
Training loss is the average cross-entropy (or other) loss computed on the training set during training, measuring how well the model fits the data it is learning from.
Where is it used?
Every LLM training run logs training loss via `Trainer` and `wandb`; it should decrease smoothly — divergence indicates learning-rate or data issues.
How to build it
Log `loss = outputs.loss.item()` each step in the training loop, plot with `wandb.log({"train/loss": loss})`, and inspect the curve for spikes or plateau.