How does early stopping work?
Training runs in rounds, and after each round the model is scored on validation data it does not train on. The loop keeps the best score and the weights that produced it. Each round without improvement adds one to a counter; when the counter reaches the patience setting, training ends. The Deep Learning textbook calls it “probably the most commonly used form of regularization in deep learning” (Goodfellow, Bengio and Courville, section 7.8, read 16 November 2026). Unlike other regularization, it changes nothing in the loss function, and for a simple linear model with a quadratic error, trained with gradient descent, the same chapter shows it is equivalent to an L2 penalty. Early stopping turns the number of training steps into a hyperparameter chosen on validation data.
What is patience in early stopping?
Patience is how many checks without improvement training tolerates before it stops. min_delta is the smallest change that counts as an improvement. A check is not always an epoch: in PyTorch Lightning, patience “counts the number of validation checks with no improvement, and not the number of training epochs”, so check_val_every_n_epoch=10 with patience=3 runs at least 40 epochs (Lightning 2.6.6 docs, read 16 November 2026). No patience value suits every model. Set patience from how noisy the validation curve is: a jumpy metric needs more checks before a plateau is real.
How do Keras, PyTorch and other libraries implement it?
Most training libraries ship early stopping as a callback or flag with its own defaults.
| Library | API | Default patience | Detail to know |
|---|---|---|---|
| Keras 3 | EarlyStopping, monitors val_loss |
0 | restore_best_weights=False |
| PyTorch Lightning 2.6.6 | EarlyStopping, monitor required |
3 checks | Counts validation checks, not epochs |
| Hugging Face Transformers 5.23 | EarlyStoppingCallback |
1 evaluation | Needs load_best_model_at_end and metric_for_best_model |
| scikit-learn 1.9.1 | MLPClassifier(early_stopping=True) |
10 epochs (n_iter_no_change) |
Off by default; holds out 10% for validation |
| XGBoost 3.4.2 | xgboost.train(early_stopping_rounds=...) |
Off (None) |
Returns the last iteration, not the best |
Defaults read 16 November 2026. Two of these libraries keep the last model, not the best one, by default: Keras unless restore_best_weights=True, and xgboost.train, whose docs point to a callback or model slicing for the best model. Without restoring, the model you ship is patience checks past its best.
When should you use early stopping?
Use it on any iterative training that has a held-out validation set: neural networks, gradient-boosted trees, and fine-tuning on a small internal dataset, where a few epochs can tip into overfitting. Two cautions. The validation set now chooses when to stop, so it is no longer an unbiased estimate; judge the final model on a separate test set, as model validation describes. And patience set too low stops a model during a temporary plateau, leaving it underfitting. Early stopping complements other regularization rather than replacing it.