Regularization
Definition
Techniques that discourage a model from becoming too complex during training, which helps it avoid overfitting.
In plain English
Think of regularization as a rule that says “keep it simple unless the data really demands otherwise.” During training, the model is rewarded for fitting the data but also lightly penalized for complexity.
Popular forms include L1 and L2 penalties on the model’s weights, dropout (randomly switching off parts of a neural network during training), and stopping training early.
Example
Adding a small penalty for large weights (L2 regularization) nudges a model toward simpler patterns instead of fitting every bump in the training data.
Draft, not yet reviewed by a person · Updated Nov 2026