TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
TorchedUp
LearnBetaProblemsSystem DesignSoonPremium
←

100. Debug Decoder I (NumPy)

Easy

The mini-decoder forward pass below was working at one point, but a refactor broke something. On the very first forward pass the function crashes with a shape-mismatch error when adding embeddings.

Find and fix the bug(s) so the function correctly maps token ids to logits.

Signature: def decoder_forward(token_ids, tok_emb, pos_emb_flat, W_out, b_out)

  • token_ids: list of length T, integer token ids in [0, vocab_size)
  • tok_emb: token-embedding weight, shape (vocab_size, d_model)
  • pos_emb_flat: positional-embedding weight as a flat list of length max_seq_len * d_model. Your code is responsible for reshaping it correctly before lookup.
  • W_out: output projection, shape (d_model, vocab_size)
  • b_out: output bias, shape (vocab_size,)

The model uses vocab_size = 8, max_seq_len = 4, d_model = 8. The pipeline is:

  1. Look up token + positional embeddings and add them
  2. Apply LayerNorm across the model dimension
  3. Project to vocabulary logits

Return the logits as a nested list of shape (T, vocab_size).

Math

ht​=LN(Etok​[xt​]+Epos​[t]),ℓt​=ht​Wout​+bout​

Related problems

  • Debug Decoder I (PyTorch)easyPyTorch

Asked at

NumPy

import numpy as np

 

def decoder_forward(...):

    pass

🔒

Premium problem

Free accounts include problems #1–20. Upgrade to unlock the editor, hidden test cases, and reference solutions for every problem.

Upgrade to PremiumBack to problems

Already premium?