What is the Naive Bayes formula?
Bayes’ theorem says the probability of a class given the features is proportional to the class’s prior probability times the likelihood of the features under that class. The “naive” step is to multiply the likelihood of each feature separately: P(class | x1, x2, …) is proportional to P(class) times P(x1 | class) times P(x2 | class), and so on. Training is just counting, how often each class occurs and how often each feature appears within it, which is why it is so fast (scikit-learn user guide, Naive Bayes).
How does a Naive Bayes spam filter work, with numbers?
Suppose 40% of mail is spam. The word “invoice” appears in 5% of spam and 20% of normal mail. A new message contains “invoice” and nothing else we track.
| Class | Prior | P(“invoice” given class) | Prior times likelihood |
|---|---|---|---|
| Spam | 0.40 | 0.05 | 0.020 |
| Not spam | 0.60 | 0.20 | 0.120 |
Normalizing, the chance of spam is 0.020 / (0.020 + 0.120), about 14%. Add more words and the filter multiplies in each word’s likelihood, so a message with several spam-leaning words quickly tips over. Real filters add a small count to every word (smoothing) so an unseen word does not zero out the whole product.
What are the types of Naive Bayes?
| Variant | Feature type | Typical use |
|---|---|---|
| Multinomial | Word counts | Document classification |
| Bernoulli | Word present or absent | Short texts |
| Gaussian | Continuous numbers, assumed bell-shaped | Sensor and tabular data |
Pick the variant by the type of feature. Text variants usually sit on a bag-of-words representation, which ignores word order and makes the independence assumption less costly.
When does Naive Bayes work, and when does it fail?
It works when features carry evidence independently, when training data is small and when you must retrain often. Its probability scores are poorly calibrated, since multiplying correlated features inflates confidence, so use the ranking, not the number, or recalibrate it. It fails when the answer depends on feature interactions, for example two words that mean something only together. Then try logistic regression or k-nearest neighbors, or a tree model. For first-pass classification of tickets, emails or documents, it is still a strong, cheap baseline to beat.