Medium

Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.

Linear Layer Weight Gradient

~15 mincode completion

Weight Gradient of a Linear Layer

A linear layer computes:

Y = X W

where $X$ has shape $(m, n)$ , $W$ has shape $(n, k)$ , and $Y$ has shape $(m, k)$ .

Given the upstream gradient (d_out, shape $(m, k)$ ), the gradient with respect to $W$ is:

Why? Each weight $W_{ij}$ affects output column $j$ through every input row. The matrix product sums those contributions over the whole batch.

Check the shapes: , this matches $W$ 's shape, which is a good sanity check.

Your task:

Implement linear_weight_grad(X, d_out) that returns .

Example Tests

2x2 input, 2x1 upstream gradient

Input: {"X":[[1,2],[3,4]],"d_out":[[1],[1]]}

Expected: [[4],[6]]

Identity input: gradient equals d_out

Input: {"X":[[1,0],[0,1]],"d_out":[[2],[3]]}

Expected: [[2],[3]]

Single sample, 3 features, 1 output

Input: {"X":[[1,2,3]],"d_out":[[1]]}

Expected: [[1],[2],[3]]

Python

import numpy as np

def linear_weight_grad(X: np.ndarray, d_out: np.ndarray) -> np.ndarray:
    """
    Compute the gradient of the loss w.r.t. the weight matrix W
    of a linear layer Y = X @ W.

    Args:
        X:     Input to the linear layer, shape (m, n)
        d_out: Upstream gradient dL/dY, shape (m, k)

    Returns:
        Weight gradient dL/dW, shape (n, k) = X.T @ d_out
    """
    # YOUR CODE HERE
    pass

Loading docs…