Optimization AlgorithmsMedium
One AdamW Step
~15 mincode completion
Implement adamw_step(theta, grad, m, v, t, lr, beta1, beta2, eps, weight_decay) returning a 2-D array whose rows are [theta_new, m_new, v_new].
Where this shows up
~15 min
••••••••••••••••
8 employers weight this skill
4 big tech firms, 2 quant funds, 1 frontier lab, 1 autonomy company. Top match scores 92.
Python
import numpy as np
def adamw_step(theta, grad, m, v, t, lr, beta1, beta2, eps, weight_decay):
"""
One AdamW update with decoupled weight decay.
Args:
theta: parameters, shape (n,)
grad: loss gradient, shape (n,)
m, v: first and second moments before this step, shape (n,)
t: step number, starting at 1
lr: learning rate eta
beta1, beta2: moment decay rates
eps: small constant in the denominator
weight_decay: lambda
Returns:
array of shape (3, n): rows theta_new, m_new, v_new
"""
# YOUR CODE HERE
pass