16. KL Divergence

Easy

Implement the KL divergence KL(P || Q).

Signature: def kl_divergence(P: np.ndarray, Q: np.ndarray) -> float

Clip Q values to avoid log(0). Both P and Q are valid probability distributions (sum to 1).

Math

D_{KL} (P ∥ Q) = i \sum P (i) lo g \frac{P ( i )}{Q ( i )}

Related problems

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○identical distributions

○P degenerate vs uniform

○non-trivial divergence🔒 Premium

○KL is non-negative

○KL is non-negative (large divergence)

○KL(P || P) = 0

○2D (B, K) batch of distributions