Underfitting in Machine Learning: Meaning, Example and Fixes

LLM foundationsModels and inferencePublished By Simon Budziak

Underfitting is when a model is too simple to learn the pattern in its training data. It scores poorly on the examples it trained on and on new data alike, the signature of high bias. The fixes run opposite to those for overfitting: more capacity, better features, longer training or less regularization.

How do you recognize underfitting?

Both scores are low and the gap between them is small. The scikit-learn guide states the rule directly: “If the training score and the validation score are both low, the estimator will be underfitting” (scikit-learn 1.9.1 validation curves guide, read 16 November 2026). A small train-test gap is not proof of a good model: it can mean the model learned too little to have anything to lose. The learning curve adds a second signal. When training and validation scores converge on a low value as the training set grows, the same guide concludes “we will probably not benefit much from more training data.” Compare against a trivial baseline as well, such as predicting the mean or the majority class: a model that barely beats it is underfitting, or its inputs carry little signal.

What is an example of underfitting?

A straight line fitted to a curve is the textbook case. The scikit-learn example fits polynomials of three degrees to 30 noisy samples of a cosine curve and scores each with 10-fold cross-validation (scikit-learn underfitting vs overfitting example, version 1.9.1, read 16 November 2026):

Polynomial degree Mean validation error (MSE) Diagnosis
1 (straight line) 0.408 Underfitting
4 0.0432 Good fit
15 1.83e+08 Overfitting

In the example’s words, the straight line “is not sufficient to fit the training samples.” The underfit line’s error is almost ten times the good fit’s, and no choice of slope or intercept can bend a straight line to a curve. The degree 15 model fails the other way, chasing every noisy point.

Overfitting vs underfitting: what’s the difference?

Underfitting fails on training and new data alike; overfitting fails only on new data.

Underfitting Good fit Overfitting
Training error High Low Very low
Validation error High Low High
Train-validation gap Small Small Large
Bias and variance High bias Balanced High variance
More data helps? Rarely A little Usually
First fix Add capacity or features Ship and monitor Regularize, simplify, add data

Overfitting memorizes noise; underfitting misses signal. Model selection is a search between the two, and it must be run on held-out data, never on the training score, using the splits that model validation describes.

How do you fix an underfitting model?

Give the model more room to learn, roughly in order of cost:

Collecting more data is the standard remedy that usually does not help, because a model that cannot fit the rows it has will not fit more of the same.

Can large language models underfit?

Large language models can underfit too, and at scale it shows up as undertraining. The Chinchilla study concluded that “current large language models are significantly undertrained”: a 70B-parameter model trained on about four times more data than the 280B-parameter Gopher, with the same compute budget, outperformed Gopher on many downstream tasks (Hoffmann et al., 2022, read 16 November 2026). The lesson carries to smaller work: check whether a model has seen enough training signal before you reach for a bigger one. The same check applies to fine-tuning, where too few steps can leave a model that still answers in its old style.

This entry was drafted with AI assistance.

Frequently asked questions

Is underfitting high bias or high variance?

High bias. An underfit model makes the same systematic error whichever training sample it sees, so its predictions are stable but consistently wrong. High variance is the overfitting side.

Does more training data fix underfitting?

Rarely. If the model cannot fit the data it already has, more rows of the same kind change little; add capacity or better features first, then check the learning curve again.

Summarize this page with

Train your team to build this