Early Stopping in Machine Learning: Patience and Defaults

LLM foundationsModels and inferencePublished By Simon Budziak

Early stopping is a training rule that halts a model once its validation score stops improving. It checks a held-out metric after each epoch or evaluation, waits a set number of checks called patience, then stops and ideally restores the best weights, which limits overfitting and saves compute.

How does early stopping work?

Training runs in rounds, and after each round the model is scored on validation data it does not train on. The loop keeps the best score and the weights that produced it. Each round without improvement adds one to a counter; when the counter reaches the patience setting, training ends. The Deep Learning textbook calls it “probably the most commonly used form of regularization in deep learning” (Goodfellow, Bengio and Courville, section 7.8, read 16 November 2026). Unlike other regularization, it changes nothing in the loss function, and for a simple linear model with a quadratic error, trained with gradient descent, the same chapter shows it is equivalent to an L2 penalty. Early stopping turns the number of training steps into a hyperparameter chosen on validation data.

What is patience in early stopping?

Patience is how many checks without improvement training tolerates before it stops. min_delta is the smallest change that counts as an improvement. A check is not always an epoch: in PyTorch Lightning, patience “counts the number of validation checks with no improvement, and not the number of training epochs”, so check_val_every_n_epoch=10 with patience=3 runs at least 40 epochs (Lightning 2.6.6 docs, read 16 November 2026). No patience value suits every model. Set patience from how noisy the validation curve is: a jumpy metric needs more checks before a plateau is real.

How do Keras, PyTorch and other libraries implement it?

Most training libraries ship early stopping as a callback or flag with its own defaults.

Library API Default patience Detail to know
Keras 3 EarlyStopping, monitors val_loss 0 restore_best_weights=False
PyTorch Lightning 2.6.6 EarlyStopping, monitor required 3 checks Counts validation checks, not epochs
Hugging Face Transformers 5.23 EarlyStoppingCallback 1 evaluation Needs load_best_model_at_end and metric_for_best_model
scikit-learn 1.9.1 MLPClassifier(early_stopping=True) 10 epochs (n_iter_no_change) Off by default; holds out 10% for validation
XGBoost 3.4.2 xgboost.train(early_stopping_rounds=...) Off (None) Returns the last iteration, not the best

Defaults read 16 November 2026. Two of these libraries keep the last model, not the best one, by default: Keras unless restore_best_weights=True, and xgboost.train, whose docs point to a callback or model slicing for the best model. Without restoring, the model you ship is patience checks past its best.

When should you use early stopping?

Use it on any iterative training that has a held-out validation set: neural networks, gradient-boosted trees, and fine-tuning on a small internal dataset, where a few epochs can tip into overfitting. Two cautions. The validation set now chooses when to stop, so it is no longer an unbiased estimate; judge the final model on a separate test set, as model validation describes. And patience set too low stops a model during a temporary plateau, leaving it underfitting. Early stopping complements other regularization rather than replacing it.

This entry was drafted with AI assistance.

Frequently asked questions

What does 3 epochs mean in early stopping?

An epoch is one full pass over the training data. A patience of 3 epochs means training stops after three consecutive evaluations with no improvement on the monitored metric.

Is early stopping a form of regularization?

Yes. It limits how far training can fit the training data, and for a simple linear model trained with gradient descent it behaves like an L2 penalty on the weights.

Summarize this page with

Train your team to build this