TorchedUp
ProblemsPremium
TorchedUp
Typical Sampling FilterMedium
ProblemsPremium

Locally Typical Sampling

Implement the locally typical filter from Meister et al. 2022.

Signature: def typical_sample_filter(probs: np.ndarray, mass: float) -> np.ndarray

  1. Compute the entropy H = -sum(p * log(p)) (use a small epsilon to avoid log(0))
  2. For each token, compute the typicality score |−log(p_i) − H| — how far its surprisal is from the average surprisal
  3. Sort tokens by typicality score ascending (most typical first)
  4. Walk that order, accumulating probability mass; always include the most-typical token, then continue including tokens until the cumulative mass first reaches or exceeds mass
  5. Zero out the rest and renormalize

Return the new probability vector.

Math

Asked at

Python (numpy)0/3 runs today

Test Results

○four-token distribution, mass=0.9
○always include most typical even if mass exceeded
○uniform distribution🔒 Premium
○filtered probabilities sum to 1
○output is non-negative
Advertisement