A complete, production-shaped PCA workflow — standardizing, choosing the number of components via a scree plot, transforming, and inverse-transforming back to check how much information was actually preserved.
The Full Workflow
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.datasets import load_digits
# 1. Dataset -- 1797 images, 64 pixel features each
data = load_digits()
X, y = data.data, data.target
# 2. Standardize -- essential before PCA
X_scaled = StandardScaler().fit_transform(X)
# 3. Fit PCA keeping ALL components first, to inspect the scree plot
pca_full = PCA()
pca_full.fit(X_scaled)
plt.plot(np.cumsum(pca_full.explained_variance_ratio_))
plt.xlabel("Number of components")
plt.ylabel("Cumulative explained variance")
plt.axhline(0.90, color="red", linestyle="--", label="90% threshold")
plt.legend()
plt.show()
# 4. Choose the number of components that reaches a target variance threshold
pca_90 = PCA(n_components=0.90) # keep enough components to explain 90% of variance
X_reduced = pca_90.fit_transform(X_scaled)
print(f"Reduced from {X.shape[1]} to {X_reduced.shape[1]} dimensions, keeping 90% of variance")Visualizing in 2D
pca_2d = PCA(n_components=2)
X_2d = pca_2d.fit_transform(X_scaled)
plt.scatter(X_2d[:,0], X_2d[:,1], c=y, cmap="tab10", alpha=0.6)
plt.xlabel(f"PC1 ({pca_2d.explained_variance_ratio_[0]:.1%} variance)")
plt.ylabel(f"PC2 ({pca_2d.explained_variance_ratio_[1]:.1%} variance)")
plt.colorbar(label="Digit")
plt.show()Reconstructing Data — Checking What Was Lost
X_reconstructed = pca_90.inverse_transform(X_reduced)
reconstruction_error = np.mean((X_scaled - X_reconstructed) ** 2)
print("Mean squared reconstruction error:", reconstruction_error)
# Visually compare an original digit image to its PCA-reconstructed version
plt.subplot(1, 2, 1); plt.imshow(X[0].reshape(8,8)); plt.title("Original")
plt.subplot(1, 2, 2); plt.imshow(pca_90.inverse_transform(X_reduced[0:1]).reshape(8,8)); plt.title("Reconstructed")
plt.show()Reconstruction lets you see exactly what information was discarded — for a well-chosen number of components, the reconstructed image should look recognizably similar to the original despite using far fewer numbers to represent it.
PCA as a Preprocessing Step in a Pipeline
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
pipeline = Pipeline([
("scaler", StandardScaler()),
("pca", PCA(n_components=0.90)),
("classifier", LogisticRegression(max_iter=1000)),
])
pipeline.fit(X_train, y_train)
print(pipeline.score(X_test, y_test))Bundling PCA inside a Pipeline ensures it's fit only on training data (per cross-validation fold, if used inside one), avoiding the same kind of data leakage risk as any other fitted preprocessing step.
Common Mistakes
- Fitting PCA on the full dataset before splitting into train/test.
- Choosing the number of components arbitrarily instead of using a scree plot or a variance-ratio threshold to justify the choice.
- Forgetting that
inverse_transform()returns data in the scaled space — you'd need to also inverse the scaler to get back to original units.
Interview Relevance
Q: "How would you decide how many principal components to keep?" Plot cumulative explained variance against the number of components (a scree plot), and choose the smallest number that reaches an acceptable threshold (commonly 90-95%) — or pass a float like n_components=0.90 directly to scikit-learn's PCA, which handles this automatically.
Practice Question
Modify the workflow above to find the minimum number of components needed to explain at least 95% of the variance in the digits dataset.
