19. Gradient Clipping

Easy

Implement gradient clipping by global norm.

Signature: def clip_gradients(grads: list, max_norm: float) -> list

Compute the global L2 norm across all gradient arrays. If it exceeds max_norm, scale all gradients by max_norm / global_norm.

Math

\overset{g}{^} = g \cdot min (1, \frac{θ}{∥ g ∥ _{2}})

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○clip needed

○no clip needed

○two grad arrays🔒 Premium

○clipped grads have norm equal to max_norm when input norm exceeds it

○idempotent when input norm ≤ max_norm (no clipping)

○list of 2D gradient arrays — global norm clip🔒 Premium