TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
←

14. Sinusoidal Positional Encoding

Medium

Implement sinusoidal positional encoding from "Attention Is All You Need".

Signature: def positional_encoding(seq_len: int, d_model: int) -> np.ndarray

Return a matrix of shape (seq_len, d_model) where:

  • Even indices: PE[pos, 2i] = sin(pos / 10000^(2i/d_model))
  • Odd indices: PE[pos, 2i+1] = cos(pos / 10000^(2i/d_model))

Math

PE(pos,2i)=sin(100002i/dpos​),PE(pos,2i+1)=cos(100002i/dpos​)

Asked at

Python 30/10 runs today

Output

Anything you print() in your code will show up here after you click Run.

Test Results

○seq=1 d=4
○seq=2 d=4
○first position all zeros for sin🔒 Premium