7. Layer Normalization

Medium

Implement the layer normalization forward pass.

Signature: def layer_norm(x: np.ndarray, gamma: np.ndarray, beta: np.ndarray, eps: float = 1e-5) -> np.ndarray

Normalize across the feature dimension (last axis) for each sample independently.

Math

LN (x) = \frac{x - μ}{σ ^{2} + ϵ} \cdot γ + β

Related problems

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○identity scale/shift 1D

○2D batch

○with scale and shift🔒 Premium

○shift-invariant: ln(x) == ln(x + c) (per-row mean subtracted)

○per-row output sums to 0 with identity gamma/beta (1D)

○3D (B, T, C) transformer input — normalize along last axis