MDSection 15 of 38 · 9 subtopics
LLM Architecture
Parameter counts, weight matrices, depth vs width, head count. The blueprint of real LLMs.
Learning path
Work through each subtopic in order. Click any file below to open its lesson. Track your progress with the checkbox at the bottom.
Subtopics
Each subtopic includes a concise explanation, code examples, and references.
01
Model Parameters
Parameter map.
model-parameters
02
Parameter Count
Parameter calculator.
parameter-count
03
Weight Matrices
Matrix explorer.
weight-matrices
04
Embedding Parameters
Vocabulary × embedding-dimension calculator.
embedding-parameters
05
Attention Parameters
Q/K/V/output projection calculator.
attention-parameters
06
Feed-Forward Parameters
FFN parameter calculator.
feed-forward-parameters
07
Model Depth
Layer-stack visualization.
model-depth
08
Model Width
Hidden-dimension visualization.
model-width
09
Number of Heads
Head configuration control.
number-of-heads