Hard

RMSprop Update

Hard

~20 min

code completion

RMSprop

RMSprop adapts the learning rate for each parameter based on recent gradient magnitudes. Parameters with large gradients get smaller effective learning rates, and vice versa:

s \leftarrow β s + (1 - β) g^{2}

w \leftarrow w - \frac{α}{s + ϵ} g

where:

s

— running average of squared gradients (per-parameter)

β

— decay rate (typically 0.9)

ϵ

— small constant for numerical stability (e.g. 1e-8)

g

— current gradient

Your task:

Implement rmsprop_update(weights, grad_sq, gradient, learning_rate, beta, epsilon). Return (weights_new, grad_sq_new).

Example Tests

Weights after update, cold start beta=0 (accessor [0])

Input: {"beta":0,"epsilon":0,"grad_sq":[0],"weights":[1],"gradient":[2],"learning_rate":0.1}

Expected: [0.9]

Weights: warm grad_sq normalizes step (accessor [0])

Input: {"beta":0,"epsilon":0,"grad_sq":[9],"weights":[0],"gradient":[3],"learning_rate":0.1}

Expected: [-0.1]

grad_sq_new accumulates (accessor [1])

Input: {"beta":0.9,"epsilon":0,"grad_sq":[0],"weights":[0],"gradient":[4],"learning_rate":0.1}

Expected: [1.6]

You can read the full problem statement above. Create a free account to run code in the browser, submit solutions, and track your progress.