What Is Naive Bayes? Formula, Example and When to Use It

LLM foundationsModels and inferencePublished By Simon Budziak

Naive Bayes is a family of simple classifiers that apply Bayes' theorem with an independence assumption between features. The assumption is rarely true, yet the method trains in one pass, needs little data and works well for text such as spam filtering. Each feature is treated as independent once the class is known.

What is the Naive Bayes formula?

Bayes’ theorem says the probability of a class given the features is proportional to the class’s prior probability times the likelihood of the features under that class. The “naive” step is to multiply the likelihood of each feature separately: P(class | x1, x2, …) is proportional to P(class) times P(x1 | class) times P(x2 | class), and so on. Training is just counting, how often each class occurs and how often each feature appears within it, which is why it is so fast (scikit-learn user guide, Naive Bayes).

How does a Naive Bayes spam filter work, with numbers?

Suppose 40% of mail is spam. The word “invoice” appears in 5% of spam and 20% of normal mail. A new message contains “invoice” and nothing else we track.

Class Prior P(“invoice” given class) Prior times likelihood
Spam 0.40 0.05 0.020
Not spam 0.60 0.20 0.120

Normalizing, the chance of spam is 0.020 / (0.020 + 0.120), about 14%. Add more words and the filter multiplies in each word’s likelihood, so a message with several spam-leaning words quickly tips over. Real filters add a small count to every word (smoothing) so an unseen word does not zero out the whole product.

What are the types of Naive Bayes?

Variant Feature type Typical use
Multinomial Word counts Document classification
Bernoulli Word present or absent Short texts
Gaussian Continuous numbers, assumed bell-shaped Sensor and tabular data

Pick the variant by the type of feature. Text variants usually sit on a bag-of-words representation, which ignores word order and makes the independence assumption less costly.

When does Naive Bayes work, and when does it fail?

It works when features carry evidence independently, when training data is small and when you must retrain often. Its probability scores are poorly calibrated, since multiplying correlated features inflates confidence, so use the ranking, not the number, or recalibrate it. It fails when the answer depends on feature interactions, for example two words that mean something only together. Then try logistic regression or k-nearest neighbors, or a tree model. For first-pass classification of tickets, emails or documents, it is still a strong, cheap baseline to beat.

This entry was drafted with AI assistance.

Frequently asked questions

When should Naive Bayes be used?

Use it as a fast first baseline for text classification, with little training data, many features or a need to retrain constantly. Move on if feature interactions drive the answer.

What is the difference between KNN and Naive Bayes?

Naive Bayes learns class probabilities from training counts and is fast to predict. K-nearest neighbors stores all examples and compares a new one to its closest neighbors, so prediction slows as data grows.

Summarize this page with

Train your team to build this