Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.
Partial Derivatives and the Gradient
A model has thousands of parameters, so "the derivative" is not one number. A partial derivative varies one input and freezes the rest:
Stack all of them into a vector and you have the gradient:
The gradient always has the same shape as the thing you are differentiating with respect to. That is not a convention, it is a constraint, and it is the single most useful check in backpropagation: if your gradient's shape does not match its parameter, a transpose is in the wrong place.
Geometrically the gradient points in the direction of steepest increase, which is why gradient descent subtracts it.
This problem uses f(x,y)=x2y+y3:
at (x, y) = (2, 1): df/dx = 2*2*1 = 4 df/dy = 4 + 3 = 7 gradient = [4, 7]
Notice that still contains y. Freezing a variable does not remove it — it just stops it from varying.
Your task:
Implement gradient_at(x, y) returning as a two-element list, using the analytic formulas above.
Example Tests
At (2, 1) the partials are 4 and 7
Input: {"x":2,"y":1}
Expected: [4,7]
On the y axis the x partial vanishes but the y partial does not
Input: {"x":0,"y":2}
Expected: [0,12]
At the origin the surface is flat to first order
Input: {"x":0,"y":0}
Expected: [0,0]
def gradient_at(x, y):
"""
Analytic gradient of f(x, y) = x^2*y + y^3.
Args:
x: first coordinate
y: second coordinate
Returns:
list of two floats: [df/dx, df/dy]
"""
# YOUR CODE HERE
pass