A complete, production-shaped KNN workflow in Python — including the two steps that matter most for KNN specifically and are easy to skip: feature scaling, and tuning k with cross-validation.
The Full Workflow
import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer
# 1. Dataset
data = load_breast_cancer()
X, y = data.data, data.target
# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# 3. Pipeline -- scaling and model bundled together (critical for KNN specifically)
pipeline = Pipeline([
("scaler", StandardScaler()),
("knn", KNeighborsClassifier()),
])
# 4. Tune k (and optionally the weighting scheme) via cross-validation
param_grid = {
"knn__n_neighbors": [3, 5, 7, 9, 11],
"knn__weights": ["uniform", "distance"],
}
grid_search = GridSearchCV(pipeline, param_grid, cv=5, scoring="accuracy")
grid_search.fit(X_train, y_train)
print("Best params:", grid_search.best_params_)
print("Best CV accuracy:", grid_search.best_score_)
# 5. Evaluate the best model on the held-out test set
best_model = grid_search.best_estimator_
predictions = best_model.predict(X_test)
print(classification_report(y_test, predictions))Why the Pipeline matters here specifically: wrapping the scaler and the KNN model together ensures the scaler is refit correctly within each cross-validation fold, using only that fold's training data — preventing the exact kind of data leakage that manually scaling once, upfront, would risk.
Visualizing How Accuracy Changes with k
import matplotlib.pyplot as plt
from sklearn.model_selection import cross_val_score
k_values = range(1, 21)
scores = []
for k in k_values:
pipe = Pipeline([("scaler", StandardScaler()), ("knn", KNeighborsClassifier(n_neighbors=k))])
cv_scores = cross_val_score(pipe, X_train, y_train, cv=5)
scores.append(cv_scores.mean())
plt.plot(k_values, scores, marker="o")
plt.xlabel("k"); plt.ylabel("Cross-validated accuracy")
plt.show()See the U-shaped curve described in Choosing k in KNN — this plot is exactly how you'd discover it on real data.
KNN for Regression — The Same Pattern
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_absolute_error
reg_pipeline = Pipeline([
("scaler", StandardScaler()),
("knn", KNeighborsRegressor(n_neighbors=5, weights="distance")),
])
reg_pipeline.fit(X_train_reg, y_train_reg)
preds = reg_pipeline.predict(X_test_reg)
print(mean_absolute_error(y_test_reg, preds))Common Mistakes
- Scaling the full dataset before the train/test split, or before cross-validation folds — always scale inside a Pipeline so it's refit per fold.
- Leaving
n_neighborsat its scikit-learn default (5) without ever tuning it for the specific dataset. - Forgetting that GridSearchCV's parameter names need the
knn__prefix when the estimator is a named step inside a Pipeline.
Interview Relevance
Q: "Why wrap KNN in a Pipeline with a scaler instead of scaling the data once beforehand?" So that scaling gets refit correctly inside every cross-validation fold using only that fold's training data — scaling once on the full dataset beforehand leaks information about validation/test folds into the fitted scaler.
Practice Question
Modify the grid search above to also tune the distance metric (knn__p, where 1=Manhattan and 2=Euclidean) alongside n_neighbors.
