Medium

Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.

The Hessian

~14 mincode completion

The Hessian collects every second partial derivative:

For a function of $n$ variables it is $n \times n$ , and it is symmetric whenever the second derivatives are continuous — differentiating with respect to $x$ then $y$ gives the same answer as $y$ then $x$ . That is a free correctness check on any Hessian you compute: if it is not symmetric, you have made an arithmetic error.

The Hessian is the multivariable curvature, and it classifies stationary points by its eigenvalues:

all positive — a minimum, curving up in every direction.

all negative — a maximum.

mixed signs — a saddle: up in one direction, down in another.

Saddles are the reason high-dimensional optimisation is hard. In a thousand dimensions, for a stationary point to be a true minimum every one of a thousand eigenvalues must be positive — overwhelmingly unlikely by chance. Almost every point where the gradient vanishes is a saddle, and escaping them, not finding minima, is what momentum is really for.

The ratio of the largest to the smallest eigenvalue is the condition number, and it is why unscaled features train badly: a long thin valley forces a learning rate small enough for the steep direction, which is far too small for the shallow one.

This problem uses $f (x, y) = x^{2} y + y^{3}$ again:

H = [2 y 2 x 2 x 6 y]

at (2, 1):  H = [[2, 4],
                 [4, 6]]     symmetric, as it must be

Your task:

Implement hessian_at(x, y) returning the $2 \times 2$ Hessian as a nested list.

Example Tests

At (2, 1) both off-diagonal entries are 4, as symmetry requires

Input: {"x":2,"y":1}

Expected: [[2,4],[4,6]]

At the origin the Hessian is all zeros: flat to second order too

Input: {"x":0,"y":0}

Expected: [[0,0],[0,0]]

Negative y curves the surface downward along both diagonal entries

Input: {"x":1,"y":-1}

Expected: [[-2,2],[2,-6]]

Python

def hessian_at(x, y):
    """
    Hessian of f(x, y) = x^2*y + y^3.

    Args:
        x: first coordinate
        y: second coordinate

    Returns:
        2x2 nested list, symmetric
    """
    # YOUR CODE HERE
    pass

Loading docs…