MDSection 25 of 38 · 6 subtopics
Alignment
Making models helpful and safe. RLHF, reward models, PPO, and DPO.
Learning path
Work through each subtopic in order. Click any file below to open its lesson. Track your progress with the checkbox at the bottom.
Subtopics
Each subtopic includes a concise explanation, code examples, and references.
01
Human Preferences
Pairwise response comparison.
human-preferences
02
Preference Data
Preference dataset table.
preference-data
03
Reward Model
Preference → reward visualization.
reward-model
04
RLHF
Complete RLHF pipeline.
rlhf
05
PPO
Policy-update visualization.
ppo
06
DPO
Chosen vs rejected probability comparison.
dpo