MDSection 10 of 38 · 6 subtopics
Multi-Head Attention
Parallel attention heads focusing on different relationships. Projections, concatenation, output.
Learning path
Work through each subtopic in order. Click any file below to open its lesson. Track your progress with the checkbox at the bottom.
Subtopics
Each subtopic includes a concise explanation, code examples, and references.
01
Attention Head
Individual-head visualizer.
attention-head
02
Multiple Attention Heads
Parallel attention panels.
multiple-attention-heads
03
Head Dimension
Dimension calculator.
head-dimension
04
QKV Projection
Projection pipeline.
qkv-projection
05
Concatenation
Head-output stacking visualization.
concatenation
06
Output Projection
Final linear transformation.
output-projection