TorchedUp
ProblemsPremium
TorchedUp
Sinusoidal Positional EncodingMedium
ProblemsPremium

Sinusoidal Positional Encoding

Implement sinusoidal positional encoding from "Attention Is All You Need".

Signature: def positional_encoding(seq_len: int, d_model: int) -> np.ndarray

Return a matrix of shape (seq_len, d_model) where:

  • Even indices: PE[pos, 2i] = sin(pos / 10000^(2i/d_model))
  • Odd indices: PE[pos, 2i+1] = cos(pos / 10000^(2i/d_model))

Math

Asked at

Python (numpy)0/3 runs today

Test Results

○seq=1 d=4
○seq=2 d=4
○first position all zeros for sin🔒 Premium
Advertisement