TorchedUp
ProblemsPremium
TorchedUp
Expected Calibration ErrorMedium
ProblemsPremium

Expected Calibration Error (ECE)

Given per-sample predicted-class confidences, predicted classes, and true labels, compute ECE with equal-width binning.

Signature: def ece(confidences: list, predictions: list, labels: list, n_bins: int = 10) -> float

  • Split [0, 1] into n_bins equal-width bins.
  • For each bin: gap = | accuracy - mean_confidence | * (bin_count / N)
  • Sum gaps across bins. Empty bins contribute 0.

The last bin should be inclusive on the right edge (so confidence == 1.0 falls into the last bin, not outside).

Math

Asked at

Python (numpy)0/3 runs today

Test Results

○5 bins basic
○all same bin, 50% accurate
○10 bins🔒 Premium
Advertisement