A cross-entropy classifier that has not learned anything outputs a uniform distribution, giving loss log(K) where K is the number of classes. Detect this state.
Signature: def is_dead_network(loss_history: list, num_classes: int, tol: float = 0.05) -> bool
Return True if the mean of the last N = min(20, len(loss_history)) losses is within tol of log(num_classes) (in absolute value).
Math
Asked at
Test Results