Multinomial Naive Bayes handles count-based features — most commonly word frequencies in text — and is the standard variant behind classical spam filters and document classifiers.
Formula
\(\text{count}(x_i,y)\) is how many times word/feature \(i\) appears across all class-\(y\) training documents. \(\text{count}(y)\) is the total word count across all class-\(y\) documents. \(n\) is the vocabulary size (number of distinct features). \(\alpha\) is the Laplace smoothing constant — without it, any word that never appeared in a class's training data would give that class a probability of exactly zero, wiping out the entire prediction regardless of every other word's evidence.
Worked Example
Training data: 2 spam emails ("free money free offer", "free win money now") and 2 ham emails ("meeting schedule tomorrow", "project meeting update"). Vocabulary (10 unique words across both classes): free, money, offer, win, now, meeting, schedule, tomorrow, project, update.
| Class | Word Counts | Total Words |
|---|---|---|
| Spam | free:3, money:2, offer:1, win:1, now:1 | 8 |
| Ham | meeting:2, schedule:1, tomorrow:1, project:1, update:1 | 6 |
With \(\alpha=1\) (Laplace smoothing), \(n=10\):
New email: "free money" — priors \(P(\text{spam})=P(\text{ham})=0.5\).
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer
docs = ["free money free offer", "free win money now", "meeting schedule tomorrow", "project meeting update"]
labels = [1, 1, 0, 0] # 1 = spam, 0 = ham
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(docs)
model = MultinomialNB(alpha=1.0) # alpha=1.0 is Laplace smoothing, applied automatically
model.fit(X, labels)
new_email = vectorizer.transform(["free money"])
print(model.predict_proba(new_email)) # closely matches the ~0.905 hand calculationWhy Laplace Smoothing Is Non-Negotiable Here
Without \(\alpha\), \(P(\text{offer}\mid\text{ham}) = 0/6 = 0\) exactly — and since Naive Bayes multiplies probabilities together, a single zero collapses the entire product to zero, regardless of how strongly every other word points toward "ham." Smoothing guarantees every feature gets a small, non-zero probability under every class, so one unseen word can't single-handedly override all other evidence.
Practical Use Cases
- Spam filtering, document/topic classification, sentiment analysis
- Any classification task with Bag-of-Words or TF-IDF style count features
Common Mistakes
- Forgetting smoothing (\(\alpha=0\)), causing unseen words to zero out entire class probabilities.
- Using Multinomial NB on raw binary presence/absence features instead of true counts — see Bernoulli Naive Bayes for that specific case.
Interview Relevance
Q: "What problem does Laplace smoothing solve in Multinomial Naive Bayes?" It prevents a word that never appeared in a class's training data from giving that class exactly zero probability — since Naive Bayes multiplies per-feature probabilities together, one zero would otherwise override every other word's evidence entirely.
Practice Question
Using the worked example's word counts, compute \(P(\text{meeting}\mid\text{spam})\) with Laplace smoothing (\(\alpha=1\), \(n=10\)).
