Medium

Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.

Second Derivatives and Curvature

~12 mincode completion

The first derivative says which way is downhill. The second says how quickly that direction is changing — the curvature.

That $- 2 f (x)$ in the middle is not arbitrary: it is the difference of two first differences, one either side, which is what "rate of change of the rate of change" means when you write it out.

Curvature is what decides whether a stationary point is a minimum:

f^{''} > 0

— curving upward, a minimum. The surface holds you.

f^{''} < 0

— curving downward, a maximum. The surface sheds you.

f^{''} = 0

— flat to second order; a saddle or an inflection, and the reason high-dimensional optimisation is hard.

It also sets the largest learning rate that will not diverge. For a quadratic with curvature $f^{''}$ , gradient descent is stable only while — a sharply curved valley demands small steps, and exceeding that bound is exactly the oscillating loss you get when the learning rate is too high.

f(x) = x^2      f''(x) = 2 everywhere   -> stable while eta < 1
f(x) = 5x^2     f''(x) = 10             -> stable while eta < 0.2

Your task:

Implement second_derivative(coeffs, x, h) using the formula above, with coeffs given low power first as in the earlier problem.

Note the $h^{2}$ in the denominator: use a larger $h$ than you would for a first derivative, because dividing noise by $h^{2}$ amplifies it.

Example Tests

x squared curves upward at a constant rate of 2

Input: {"h":0.001,"x":3,"coeffs":[0,0,1]}

Expected: 2

A straight line has no curvature at all

Input: {"h":0.001,"x":10,"coeffs":[4,2]}

Expected: 0

A steeper parabola curves ten times as hard, so tolerates a fifth of the learning rate

Input: {"h":0.001,"x":1,"coeffs":[0,0,5]}

Expected: 10

Python

import numpy as np


def second_derivative(coeffs, x, h):
    """
    Numerical second derivative of a polynomial.

    Args:
        coeffs: coefficients low power first, so [1, 0, 2] is 1 + 2x^2
        x:      the point to evaluate at
        h:      the step size

    Returns:
        float: (f(x+h) - 2f(x) + f(x-h)) / h^2
    """
    # YOUR CODE HERE
    pass

Loading docs…