Medium

Softmax Activation

Medium

~15 min

code completion

Softmax

Softmax converts a vector of raw scores (logits) into a probability distribution that sums to 1:

softmax (z)_{i} = \frac{e ^{z_{i}}}{\sum _{j} e ^{z_{j}}}

It's used in the output layer of multi-class classifiers.

Numerical stability: large $z$ values cause exp(z) to overflow. The standard fix is to subtract the maximum before exponentiating:

softmax (z) = \frac{e ^{z - m a x (z)}}{\sum _{j} e ^{z_{j} - m a x (z)}}

This leaves the output unchanged mathematically but avoids overflow.

Your task:

Implement softmax(z) using the numerically stable formula.

Example Tests

3 logits

Input: {"z":[1,2,3]}

Expected: [0.09003,0.24473,0.66524]

Equal logits: uniform distribution

Input: {"z":[0,0,0]}

Expected: [0.33333,0.33333,0.33333]

Output sums to 1

Input: {"z":[1,2,3,4]}

Expected: 1

You can read the full problem statement above. Create a free account to run code in the browser, submit solutions, and track your progress.