Attention Context Vector
~10 mincode completion
Implement attention_context(weights, V) that returns .
Examples
Single query attending fully to first value row
- Input
- attention_context([1, 0, 0], [[1, 2], [3, 4], [5, 6]])
- Output
- [1, 2]
Equal weights over two value rows: midpoint
- Input
- attention_context([0.5, 0.5], [[2, 4], [6, 8]])
- Output
- [4, 6]
Uniform weights: output is mean of value rows
- Input
- attention_context([0.3333333333333333, 0.3333333333333333, 0.3333333333333333], [[3, 6], [6, 3], [9, 3]])
- Output
- [6, 4]
Hints
Hint 1
Use a matrix product rather than nested loops, and check which operand transposes.
Hint 2
Make sure you are not returning V unchanged.
Requirements
weights: Attention weights, shape (T_q, T_k) or (T_k,) for a single queryV: Value matrix, shape (T_k, d_v)Return Context vector(s): weights @ V, shape (T_q, d_v) or (d_v,)
Constraints
Allowed library: NumPy only
Time limit: 200 ms, Memory: 64 MB
Where this shows up
~10 min
••••••••••••••••
8 employers weight this skill
4 frontier labs, 3 AI product companies, 1 enterprise vendor. Top match scores 93.
Python
import numpy as np
def attention_context(weights: np.ndarray, V: np.ndarray) -> np.ndarray:
"""
Compute attention context vector(s).
Args:
weights: Attention weights, shape (T_q, T_k) or (T_k,) for a single query
V: Value matrix, shape (T_k, d_v)
Returns:
Context vector(s): weights @ V, shape (T_q, d_v) or (d_v,)
"""
# YOUR CODE HERE
pass