How do you recognize underfitting?
Both scores are low and the gap between them is small. The scikit-learn guide states the rule directly: “If the training score and the validation score are both low, the estimator will be underfitting” (scikit-learn 1.9.1 validation curves guide, read 16 November 2026). A small train-test gap is not proof of a good model: it can mean the model learned too little to have anything to lose. The learning curve adds a second signal. When training and validation scores converge on a low value as the training set grows, the same guide concludes “we will probably not benefit much from more training data.” Compare against a trivial baseline as well, such as predicting the mean or the majority class: a model that barely beats it is underfitting, or its inputs carry little signal.
What is an example of underfitting?
A straight line fitted to a curve is the textbook case. The scikit-learn example fits polynomials of three degrees to 30 noisy samples of a cosine curve and scores each with 10-fold cross-validation (scikit-learn underfitting vs overfitting example, version 1.9.1, read 16 November 2026):
| Polynomial degree | Mean validation error (MSE) | Diagnosis |
|---|---|---|
| 1 (straight line) | 0.408 | Underfitting |
| 4 | 0.0432 | Good fit |
| 15 | 1.83e+08 | Overfitting |
In the example’s words, the straight line “is not sufficient to fit the training samples.” The underfit line’s error is almost ten times the good fit’s, and no choice of slope or intercept can bend a straight line to a curve. The degree 15 model fails the other way, chasing every noisy point.
Overfitting vs underfitting: what’s the difference?
Underfitting fails on training and new data alike; overfitting fails only on new data.
| Underfitting | Good fit | Overfitting | |
|---|---|---|---|
| Training error | High | Low | Very low |
| Validation error | High | Low | High |
| Train-validation gap | Small | Small | Large |
| Bias and variance | High bias | Balanced | High variance |
| More data helps? | Rarely | A little | Usually |
| First fix | Add capacity or features | Ship and monitor | Regularize, simplify, add data |
Overfitting memorizes noise; underfitting misses signal. Model selection is a search between the two, and it must be run on held-out data, never on the training score, using the splits that model validation describes.
How do you fix an underfitting model?
Give the model more room to learn, roughly in order of cost:
- Better inputs: feature engineering often fixes a linear model faster than a bigger model does: a squared term, a ratio or a day-of-week flag can expose a pattern the model could not express.
- More capacity: more polynomial terms, deeper trees, more layers, or boosting, which trains models in sequence to cut bias (see bagging vs boosting).
- Less regularization: a penalty set too high in regularization flattens the weights until the pattern disappears.
- Longer training: if training loss is still falling when the run ends, it stopped too soon; early stopping should fire on a validation plateau, not after a fixed small number of epochs.
- Optimizer settings: a learning rate set too low in gradient descent can leave the loss high at the end of training and look like underfitting.
Collecting more data is the standard remedy that usually does not help, because a model that cannot fit the rows it has will not fit more of the same.
Can large language models underfit?
Large language models can underfit too, and at scale it shows up as undertraining. The Chinchilla study concluded that “current large language models are significantly undertrained”: a 70B-parameter model trained on about four times more data than the 280B-parameter Gopher, with the same compute budget, outperformed Gopher on many downstream tasks (Hoffmann et al., 2022, read 16 November 2026). The lesson carries to smaller work: check whether a model has seen enough training signal before you reach for a bigger one. The same check applies to fine-tuning, where too few steps can leave a model that still answers in its old style.