The margin is the width of the empty buffer zone between the decision boundary and the nearest points of either class — SVM's entire training objective is to make this width as large as possible.
Formula
\(\lVert w \rVert\) is the magnitude of the weight vector defining the decision boundary \(w^Tx+b=0\). This formula is why minimizing \(\lVert w \rVert\) (equivalently, \(\tfrac{1}{2}\lVert w \rVert^2\), a mathematically convenient equivalent form) directly maximizes the margin — smaller \(\lVert w \rVert\) means a larger margin.
Where the "2" and the "1" Come From
SVM defines the two margin boundary lines as \(w^Tx+b=+1\) and \(w^Tx+b=-1\) — chosen (as a scaling convention) so the distance between these two parallel lines works out cleanly to \(2/\lVert w \rVert\). This is a standard normalization, not an arbitrary constant — it makes the constraint \(y_i(w^Tx_i+b)\geq 1\) in the optimization problem exactly capture "every point is on the correct side of its margin boundary."
Worked Numerical Example
For \(w=[2,0]\), \(b=-4\): the decision boundary is \(2x_1-4=0\), i.e. \(x_1=2\).
Checking two candidate support vectors: for \((2.5, 0)\), class \(+1\): \(w^Tx+b = 2(2.5)-4 = 1\) — exactly on the \(+1\) margin line. For \((1.5, 0)\), class \(-1\): \(2(1.5)-4=-1\) — exactly on the \(-1\) margin line. The gap between \(x_1=2.5\) and \(x_1=1.5\) is exactly \(1\), matching the margin formula.
import numpy as np
w = np.array([2, 0])
b = -4
for point, label in [((2.5,0), 1), ((3,1), 1), ((1.5,0), -1), ((1,1), -1)]:
functional_margin = label * (np.dot(w, point) + b)
print(point, label, functional_margin)
# (2.5, 0) 1 -> 1.0 (support vector -- exactly on the margin)
# (3, 1) 1 -> 2.0 (safely inside, not a support vector)
# (1.5, 0) -1 -> 1.0 (support vector -- exactly on the margin)
# (1, 1) -1 -> 2.0 (safely inside, not a support vector)
margin = 2 / np.linalg.norm(w)
print(margin) # 1.0Why Maximizing the Margin Improves Generalization
A wide margin means the boundary has "room to spare" — small perturbations in new data (measurement noise, slightly different feature values than any training example) are much less likely to cross a wide margin than a narrow one. This is a concrete, geometric version of the general principle that simpler, more robust decision boundaries tend to generalize better than boundaries that fit the training data as tightly as possible.
Practical Use Cases
- Understanding exactly what SVM's training process is optimizing for, beyond "find a separating line"
- Reasoning about the effect of the
Chyperparameter — smaller C explicitly favors a wider margin over training accuracy
Common Mistakes
- Confusing the margin (the width of the buffer zone, \(2/\lVert w \rVert\)) with the decision boundary itself (the single line \(w^Tx+b=0\) at the center of that zone).
- Assuming a wider margin always means better accuracy on the training set — it's specifically a bet on better generalization, sometimes at the cost of some training accuracy (see the soft margin and C).
Interview Relevance
Q: "Why does minimizing ||w|| maximize the margin?" Because the margin is defined as \(2/\lVert w \rVert\) — an inverse relationship — so making \(\lVert w \rVert\) as small as possible (subject to the classification constraints) directly maximizes the margin width.
Practice Question
If \(w=[3,4]\) and \(b=-10\), compute \(\lVert w \rVert\) and the resulting margin.
