4. Cross-Entropy Loss

Easy

Implement categorical cross-entropy loss.

Signature: def cross_entropy(y_pred: np.ndarray, y_true: np.ndarray) -> float

Clip predictions to avoid log(0).

Math

L = - i \sum y_{i} lo g (\overset{y}{^}_{i} + ϵ)

Related problems

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○perfect prediction

○typical

○log stability🔒 Premium

○loss is non-negative

○loss is non-negative (uniform pred)

○zero loss for one-hot match

○3D (B, T, V) per-token one-hot targets