AWS Builder Center

Functional vs Geometric Margins in SVMs: Why One of Them Lies to You

When I started learning SVMs (CS229, Lecture 7), the idea of "margin" sounded simple — the gap between the decision boundary and the closest points. But there are actually two different definitions of margin, and confusing them is exactly what got me stuck.

Machine Learning
Functional Margin
For a hyperplane defined by weights  w  and bias  b , the functional margin of a single training example  (x, y)  is:
γ̂ = y(wᵀx + b)
This tells you two things: which side of the boundary the point is on, and roughly how confident the classifier is (the sign should match  y , and a bigger magnitude means more confidence).
The flaw: the functional margin has no fixed scale. If you multiply  w  and  b  both by 2, the geometric position of the decision boundary doesn't change at all — but the functional margin doubles. So functional margin can be made arbitrarily large just by scaling  w , without actually improving the classifier. It's not a meaningful measure of "how good" the boundary is on its own.
Geometric Margin
This fixes the scaling problem by normalizing:
γ = y(wᵀx + b) / ||w||
Dividing by ||w|| makes the margin scale-invariant — multiply  w  and  b  by any positive constant, and the geometric margin stays the same. This is the actual Euclidean distance from the point to the hyperplane, which is what we intuitively mean by "margin."
Why this matters for SVMs
The whole point of an SVM is to find the hyperplane that maximizes the geometric margin to the closest points (the support vectors). If we tried to maximize the functional margin instead, the optimization would be ill-posed — it could just keep scaling  w  upward forever with no real improvement, never converging to a meaningful answer.
The fix in practice
To make the optimization well-behaved, SVMs typically fix the functional margin of the closest point to 1 (a normalization constraint), which turns maximizing the geometric margin into the equivalent, cleaner problem of minimizing ||w|| subject to every point satisfying y(wᵀx + b) ≥ 1. This is what leads to the classic SVM optimization formulation.
Takeaway
The confusion between functional and geometric margin is really a lesson in why scale-invariance matters in optimization — a quantity that can be inflated by scaling isn't something you should be maximizing directly. Geometric margin fixes that by tying the measure back to actual distance, which is what makes the SVM's optimization problem solvable in the first place.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article