59. Debug: Attention Scale

Easy

The function below is supposed to compute scaled dot-product attention, but it has a bug. Find and fix it.

Signature: def buggy_attention(Q, K, V)

Hint: Compare the implementation to the standard scaled dot-product attention formula and check the normalization step before the softmax.

Math

scaled dot-product attention: Attention (Q, K, V) = softmax (\frac{Q K ^{T}}{d _{k}}) V For unit-normal q, k \in R^{d_{k}} : E [q \cdot k] = 0, Var [q \cdot k] = d_{k}

Asked at

NumPy

import numpy as np

def buggy_attention(...):

pass

Premium problem

Free accounts include problems #1–20. Upgrade to unlock the editor, hidden test cases, and reference solutions for every problem.

Already premium?