Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.
ReLU Backward Pass
The ReLU activation function is:
Its derivative is a step function:
During backpropagation we apply the chain rule: multiply the upstream gradient d_out by this local derivative element-wise:
Gradients only flow through neurons that were active (positive) during the forward pass. Neurons where receive zero gradient, the so-called dead neuron problem with ReLU.
Your task:
Implement relu_backward(x, d_out) that returns the gradient of the loss with respect to the pre-activation input x.
Example Tests
Mixed signs: gradient flows only where x > 0
Input: {"x":[1,-1,2],"d_out":[0.5,0.5,0.5]}
Expected: [0.5,0,0.5]
All negative: gradient is zero everywhere (dead neurons)
Input: {"x":[-1,-2,-3],"d_out":[1,1,1]}
Expected: [0,0,0]
All positive: upstream gradient passes through unchanged
Input: {"x":[1,2,3],"d_out":[2,3,4]}
Expected: [2,3,4]
import numpy as np
def relu_backward(x: np.ndarray, d_out: np.ndarray) -> np.ndarray:
"""
Compute the gradient of the loss w.r.t. the input of a ReLU.
Args:
x: Pre-activation values from the forward pass, shape (n,)
d_out: Upstream gradient dL/d(ReLU(x)), shape (n,)
Returns:
Gradient dL/dx of the same shape: d_out where x > 0, else 0.
"""
# YOUR CODE HERE
pass