The mini-decoder forward pass below was working at one point, but a refactor broke something. On the very first forward pass the function crashes with a shape-mismatch error when adding embeddings.
Find and fix the bug(s) so the function correctly maps token ids to logits.
Signature: def decoder_forward(token_ids, tok_emb, pos_emb_flat, W_out, b_out)
token_ids: list of length T, integer token ids in [0, vocab_size)tok_emb: token-embedding weight, shape (vocab_size, d_model)pos_emb_flat: positional-embedding weight as a flat list of length max_seq_len * d_model. Your code is responsible for reshaping it correctly before lookup.W_out: output projection, shape (d_model, vocab_size)b_out: output bias, shape (vocab_size,)The model uses vocab_size = 8, max_seq_len = 4, d_model = 8. The pipeline is:
Return the logits as a nested list of shape (T, vocab_size).
Math
Related problems
Asked at
import numpy as np
def decoder_forward(...):
pass
Premium problem
Free accounts include problems #1–20. Upgrade to unlock the editor, hidden test cases, and reference solutions for every problem.
Already premium?