Overfitting
Overfitting occurs when a model memorizes training data patterns — including noise — and fails to generalize to unseen data. It shows up as low training loss but high validation loss.
What is Overfitting?
Overfitting occurs when a model memorizes training data patterns — including noise — and fails to generalize to unseen data. It shows up as low training loss but high validation loss.
Overfitting occurs when a model memorizes training data patterns — including noise — and fails to generalize to unseen data. It shows up as low training loss but high validation loss.
Where is it used?
Small LLM fine-tunes easily overfit narrow datasets; without regularization, deep models can memorize entire training sequences verbatim, hurting downstream evals.
How to build it
Train a high-capacity MLP on a tiny dataset, plot train vs. validation loss, and watch the gap widen — then add dropout and weight decay to close it.