Despite the name, it solves classification, not regression in the sense of predicting a quantity. The “regression” is the weighted sum underneath.
How does logistic regression turn features into a probability?
It computes a score z = b + w1x1 + w2x2 + and so on, then applies the sigmoid, p = 1 / (1 + e^-z). A score of 0 gives a probability of 0.5, large positive scores approach 1 and large negative scores approach 0. The weights are learned from labeled examples, so it is a form of supervised learning.
A worked example: predict whether a customer cancels from the number of support tickets x in a month, with learned weights b = -2 and w = 0.5.
| Tickets | z | Probability of cancelling |
|---|---|---|
| 2 | -1 | 0.27 |
| 4 | 0 | 0.50 |
| 6 | 1 | 0.73 |
Each extra ticket multiplies the odds of cancelling by e^0.5, about 1.65. That readable effect per feature is why logistic regression is still the first model many teams try.
What is the cost function in logistic regression?
The model is trained by minimizing log loss, also called binary cross-entropy. It punishes a confident wrong prediction far more than an unsure one: predicting 0.99 for a case that turns out negative costs much more than predicting 0.6. Unlike the squared error of linear regression, log loss is convex for this model, so gradient descent reaches the single best set of weights.
How does logistic regression differ from linear regression?
| Linear regression | Logistic regression | |
|---|---|---|
| Predicts | A number (a price, a duration) | A probability of a class |
| Output range | Any real value | 0 to 1 |
| Loss | Squared error | Log loss |
| Use it for | Forecasting a quantity | Spam or not, churn or not, approve or review |
Pick it when the answer is a category and you need a probability you can set a threshold on. The threshold is a business choice: lowering it from 0.5 catches more true cases and raises false alarms, the trade-off behind recall and the F1 score.
What are the limits of logistic regression?
It draws a straight boundary between classes, so it misses interactions and curves unless you add features that capture them. Strongly correlated features make individual weights unstable, and with many features it can overfit, which regularization tames. For tabular problems with clear structure it is often competitive with far heavier models, and it explains itself: every prediction is a sum of named contributions.