Naive Bayes in Python

A side-by-side comparison of all three Naive Bayes variants in scikit-learn, plus a complete tuning and evaluation workflow — so choosing the right variant for a given dataset is a concrete, code-backed decision, not a guess.

All Three Variants, Side by Side

from sklearn.naive_bayes import GaussianNB, MultinomialNB, BernoulliNB
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer

# GaussianNB -- continuous numeric features
data = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.2, random_state=42, stratify=data.target
)
gnb = GaussianNB().fit(X_train, y_train)
print("Gaussian NB accuracy:", gnb.score(X_test, y_test))

# MultinomialNB -- count-based features (e.g. word counts)
from sklearn.feature_extraction.text import CountVectorizer
docs = ["free money offer", "meeting schedule notes", "free cash prize", "project update report"]
labels = [1, 0, 1, 0]
X_text = CountVectorizer().fit_transform(docs)
mnb = MultinomialNB().fit(X_text, labels)
print("Multinomial NB fitted on", X_text.shape[1], "vocabulary features")

# BernoulliNB -- binary presence/absence features
import numpy as np
X_binary = (X_text.toarray() > 0).astype(int)   # convert counts to pure presence/absence
bnb = BernoulliNB().fit(X_binary, labels)
print("Bernoulli NB fitted on the same vocabulary, as binary features")

Choosing the Right Variant — A Decision Table

Tuning the Smoothing Parameter

from sklearn.model_selection import GridSearchCV

param_grid = {"alpha": [0.01, 0.1, 0.5, 1.0, 2.0]}
grid_search = GridSearchCV(MultinomialNB(), param_grid, cv=5, scoring="accuracy")
grid_search.fit(X_text, labels)
print("Best alpha:", grid_search.best_params_)

Alpha (smoothing) is Naive Bayes' main tunable hyperparameter — larger values smooth probabilities more aggressively toward uniform, which can help when training data is sparse relative to vocabulary size, but too much smoothing washes out genuinely useful signal.

Full Evaluation Workflow

from sklearn.metrics import classification_report, roc_auc_score

model = GaussianNB()
model.fit(X_train, y_train)

predictions = model.predict(X_test)
probabilities = model.predict_proba(X_test)[:, 1]

print(classification_report(y_test, predictions))
print("ROC-AUC:", roc_auc_score(y_test, probabilities))

Common Mistakes

• Using GaussianNB on features that are actually counts or binary — the wrong distributional assumption directly hurts likelihood estimates.
• Not tuning alpha at all, leaving it at scikit-learn's default when the dataset's vocabulary-to-data-size ratio would benefit from a different smoothing strength.
• Forgetting that mixed-type feature sets (some continuous, some count-based) don't fit cleanly into any single NB variant — this often signals it's time to reach for a different algorithm, like logistic regression or a tree-based model.

Interview Relevance

Q: "You have a dataset with both continuous and word-count features. Can you use a single Naive Bayes model directly?" Not cleanly — each NB variant assumes one specific feature distribution; a mixed feature set either needs separate NB models combined, feature-type-specific preprocessing tricks, or a different algorithm (like logistic regression or a tree-based model) that doesn't require choosing a single likelihood distribution upfront.

Practice Question

You're classifying products into categories using both a numeric "price" feature and a text "description" feature. Sketch an approach for combining these using Naive Bayes concepts, or explain why you'd switch algorithms.

Related ML Notes

• Gaussian Naive Bayes
• Naive Bayes for Text Classification
• Hyperparameter Tuning