Easy

KL Divergence

Easy

~12 min

code completion

Kullback–Leibler divergence measures how much a probability distribution $P$ differs from a reference $Q$ :

D_{K L} (P ∥ Q) = i \sum P (i) lo g \frac{P ( i )}{Q ( i )}

Key properties:

D_{K L} (P ∥ Q) \geq 0

, with equality iff

P = Q

Not symmetric:

D_{K L} (P ∥ Q) \neq = D_{K L} (Q ∥ P)

in general

Appears in VAE training (ELBO = reconstruction loss

- β D_{K L}

) and PPO policy updates

For this problem assume all values in p and q are strictly positive (no zeros).

Your task:

Implement kl_divergence(p, q) that computes $D_{K L} (P ∥ Q)$ .

Example Tests

Identical distributions: KL is zero

Input: {"p":[0.25,0.25,0.25,0.25],"q":[0.25,0.25,0.25,0.25]}

Expected: 0

Similar distributions: small KL

Input: {"p":[0.5,0.5],"q":[0.4,0.6]}

Expected: 0.02041

Very different distributions: larger KL

Input: {"p":[0.9,0.1],"q":[0.5,0.5]}

Expected: 0.36806

You can read the full problem statement above. Create a free account to run code in the browser, submit solutions, and track your progress.