ReLU Backward Pass
Implement relu_backward(x, d_out) that returns the gradient of the loss with respect to the pre-activation input x.
Examples
Mixed signs: gradient flows only where x > 0
- Input
- relu_backward([1, -1, 2], [0.5, 0.5, 0.5])
- Output
- [0.5, 0, 0.5]
All negative: gradient is zero everywhere (dead neurons)
- Input
- relu_backward([-1, -2, -3], [1, 1, 1])
- Output
- [0, 0, 0]
All positive: upstream gradient passes through unchanged
- Input
- relu_backward([1, 2, 3], [2, 3, 4])
- Output
- [2, 3, 4]
Hints
Hint 1
Work directly with the arguments x, d_out and return the result rather than printing it.
Hint 2
Watch for this: passed d out unchanged without masking.
Requirements
x: Pre-activation values from the forward pass, shape (n,)d_out: Upstream gradient dL/d(ReLU(x)), shape (n,)Return Gradient dL/dx of the same shape: d_out where x > 0, else 0.
Constraints
Allowed library: NumPy only
Time limit: 200 ms, Memory: 64 MB
Where this shows up
8 employers weight this skill
4 frontier labs, 2 big tech firms, 1 autonomy company, 1 enterprise vendor. Top match scores 93.
import numpy as np
def relu_backward(x: np.ndarray, d_out: np.ndarray) -> np.ndarray:
"""
Compute the gradient of the loss w.r.t. the input of a ReLU.
Args:
x: Pre-activation values from the forward pass, shape (n,)
d_out: Upstream gradient dL/d(ReLU(x)), shape (n,)
Returns:
Gradient dL/dx of the same shape: d_out where x > 0, else 0.
"""
# YOUR CODE HERE
pass