A partial derivative measures how a function changes with respect to just one input variable, while every other variable is held fixed. ML models have many parameters, so this — not the single-variable derivative — is what actually gets computed during training.
Formula and Notation
Numerical Example
For \(f(x,y) = x^2y + y^3\):
At \((x,y) = (2,1)\): \(\frac{\partial f}{\partial x} = 2(2)(1) = 4\), and \(\frac{\partial f}{\partial y} = (2)^2 + 3(1)^2 = 4 + 3 = 7\).
def partial_x(x, y):
return 2 * x * y
def partial_y(x, y):
return x**2 + 3 * y**2
print(partial_x(2, 1)) # 4
print(partial_y(2, 1)) # 7Why "Hold Everything Else Constant" Actually Makes Sense
Imagine standing on a hillside (a 2-variable function's surface). "How steep is it in the east-west direction?" is a different question from "how steep is it in the north-south direction?" — a partial derivative answers exactly one of those directional questions at a time, ignoring the other.
Where This Connects
Stacking every partial derivative of a function into one vector gives you the gradient: \(\nabla f = [\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}]\). A model with 100 parameters needs 100 partial derivatives — one per parameter — computed together as the gradient every training step.
Common Mistakes
- Forgetting to treat other variables as constants — accidentally applying the product rule across variables that should be held fixed.
- Computing only one partial derivative when a multi-parameter model needs the full set (the gradient) to actually update every parameter.
Interview Relevance
Q: "Why does linear regression with 5 features need 5 partial derivatives during training, not just 1?" Because each of the 5 weights is a separate parameter the model must learn, and each one needs its own partial derivative (holding the other 4 weights fixed) to know how adjusting it individually affects the loss.
Practice Question
For \(f(x,y) = 3x^2 + 2xy + y^2\), compute \(\frac{\partial f}{\partial x}\) and \(\frac{\partial f}{\partial y}\), then evaluate both at \((x,y)=(1,2)\).
