What Is Logistic Regression? Plain Definition With an Example

LLM foundationsModels and inferencePublished By Simon Budziak

Logistic regression is a supervised learning method that predicts the probability of a yes or no outcome from input features. It adds up weighted features, squashes the total into a number between 0 and 1 with the sigmoid function, and turns that probability into a class with a threshold.

Despite the name, it solves classification, not regression in the sense of predicting a quantity. The “regression” is the weighted sum underneath.

How does logistic regression turn features into a probability?

It computes a score z = b + w1x1 + w2x2 + and so on, then applies the sigmoid, p = 1 / (1 + e^-z). A score of 0 gives a probability of 0.5, large positive scores approach 1 and large negative scores approach 0. The weights are learned from labeled examples, so it is a form of supervised learning.

A worked example: predict whether a customer cancels from the number of support tickets x in a month, with learned weights b = -2 and w = 0.5.

Tickets z Probability of cancelling
2 -1 0.27
4 0 0.50
6 1 0.73

Each extra ticket multiplies the odds of cancelling by e^0.5, about 1.65. That readable effect per feature is why logistic regression is still the first model many teams try.

What is the cost function in logistic regression?

The model is trained by minimizing log loss, also called binary cross-entropy. It punishes a confident wrong prediction far more than an unsure one: predicting 0.99 for a case that turns out negative costs much more than predicting 0.6. Unlike the squared error of linear regression, log loss is convex for this model, so gradient descent reaches the single best set of weights.

How does logistic regression differ from linear regression?

Linear regression Logistic regression
Predicts A number (a price, a duration) A probability of a class
Output range Any real value 0 to 1
Loss Squared error Log loss
Use it for Forecasting a quantity Spam or not, churn or not, approve or review

Pick it when the answer is a category and you need a probability you can set a threshold on. The threshold is a business choice: lowering it from 0.5 catches more true cases and raises false alarms, the trade-off behind recall and the F1 score.

What are the limits of logistic regression?

It draws a straight boundary between classes, so it misses interactions and curves unless you add features that capture them. Strongly correlated features make individual weights unstable, and with many features it can overfit, which regularization tames. For tabular problems with clear structure it is often competitive with far heavier models, and it explains itself: every prediction is a sum of named contributions.

This entry was drafted with AI assistance.

Frequently asked questions

What is logistic regression simply?

A method that turns a weighted sum of features into a probability between 0 and 1, then calls the case positive when that probability passes a threshold, often 0.5.

What are the disadvantages of logistic regression?

It draws a linear boundary, so it misses interactions and curves unless you engineer features for them, and it needs a lot of care when features are strongly correlated.

Summarize this page with

Train your team to build this