A complete, production-shaped SVM workflow — scaling, kernel and hyperparameter selection via grid search, and reading the resulting model's support vectors.
The Full Workflow
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.svm import SVC
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer
# 1. Dataset
data = load_breast_cancer()
X, y = data.data, data.target
# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# 3. Pipeline -- scaling is essential for SVM
pipeline = Pipeline([
("scaler", StandardScaler()),
("svm", SVC()),
])
# 4. Joint grid search over kernel, C, and gamma
param_grid = [
{"svm__kernel": ["linear"], "svm__C": [0.1, 1, 10]},
{"svm__kernel": ["rbf"], "svm__C": [0.1, 1, 10], "svm__gamma": [0.001, 0.01, 0.1, 1]},
]
grid_search = GridSearchCV(pipeline, param_grid, cv=5, scoring="accuracy", n_jobs=-1)
grid_search.fit(X_train, y_train)
print("Best params:", grid_search.best_params_)
# 5. Evaluate
best_model = grid_search.best_estimator_
predictions = best_model.predict(X_test)
print(classification_report(y_test, predictions))Inspecting the Trained Model
svm_step = best_model.named_steps["svm"]
print("Number of support vectors per class:", svm_step.n_support_)
print("Total support vectors:", len(svm_step.support_vectors_))
print(f"That's {len(svm_step.support_vectors_) / len(X_train):.1%} of the training data")Multi-Class Classification
# scikit-learn handles multi-class SVM automatically via one-vs-one by default
from sklearn.datasets import load_iris
iris = load_iris()
multi_model = SVC(kernel="rbf")
multi_model.fit(iris.data, iris.target) # 3 classes -- no extra code needed
print(multi_model.predict(iris.data[:5]))SVM for Regression — SVR
from sklearn.svm import SVR
# Support Vector Regression -- same margin-based idea, applied to continuous targets:
# fit a boundary such that most points fall within a margin (epsilon) of it
reg_model = SVR(kernel="rbf", C=1.0, epsilon=0.1)
reg_model.fit(X_train_reg, y_train_reg)
predictions = reg_model.predict(X_test_reg)SVR reframes the objective: instead of maximizing the margin between classes, it fits a function such that as many points as possible fall within an \(\epsilon\)-wide tube around the prediction — points outside that tube contribute to the loss, similar in spirit to the soft-margin slack variables in classification.
Common Mistakes
- Forgetting to scale features before SVM — this alone is often the single biggest driver of a poorly-performing SVM.
- Running an unrestricted grid search over every kernel/C/gamma combination on a large dataset — SVM training time grows quickly, so start with a coarser search before refining.
- Not checking how many support vectors the final model uses — a very high fraction suggests the model may be overly complex for the chosen C/gamma.
Interview Relevance
Q: "How would you tune an SVM's hyperparameters efficiently, given training can be slow?" Start with a coarser grid search (fewer values, wider spacing) to narrow down a promising region, then refine with a finer search around the best coarse result — a two-stage search is far cheaper than one exhaustive fine-grained search from the start.
Practice Question
Modify the grid search above to also test a polynomial kernel with degrees 2 and 3.
