TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
←

181. Typical Sampling Filter

Medium

Implement the locally typical filter from Meister et al. 2022.

Signature: def typical_sample_filter(probs: np.ndarray, mass: float) -> np.ndarray

  1. Compute the entropy H = -sum(p * log(p)) (use a small epsilon to avoid log(0))
  2. For each token, compute the typicality score |−log(p_i) − H| — how far its surprisal is from the average surprisal
  3. Sort tokens by typicality score ascending (most typical first)
  4. Walk that order, accumulating probability mass; always include the most-typical token, then continue including tokens until the cumulative mass first reaches or exceeds mass
  5. Zero out the rest and renormalize

Return the new probability vector.

Math

scorei​=​−logpi​−H(p)​,H(p)=−j∑​pj​logpj​

Asked at

NumPy

import numpy as np

 

def typical_sample_filter(...):

    pass

🔒

Premium problem

Free accounts include problems #1–20. Upgrade to unlock the editor, hidden test cases, and reference solutions for every problem.

Upgrade to PremiumBack to problems

Already premium?