Training
Training is the iterative process of feeding data through a model, computing loss, calculating gradients via backprop, and updating weights with an optimizer. The goal is to minimize the loss.
What is Training?
Training is the iterative process of feeding data through a model, computing loss, calculating gradients via backprop, and updating weights with an optimizer. The goal is to minimize the loss.
Training is the iterative process of feeding data through a model, computing loss, calculating gradients via backprop, and updating weights with an optimizer. The goal is to minimize the loss.
Where is it used?
LLM pretraining runs forward, backward, and optimizer steps billions of times; fine-tuning (SFT/RLHF) trains on narrower datasets to align model behavior.
How to build it
Write a loop: `for x,y in loader: opt.zero_grad(); loss = model(x); loss.backward(); opt.step()` — this is the training loop used at every scale.