Supervised Fine-Tuning
Supervised fine-tuning (SFT) trains a pretrained model on input–output pairs (prompt → response) using teacher-forced cross-entropy loss, teaching it to follow instructions in a specific format.
What is Supervised Fine-Tuning?
Supervised fine-tuning (SFT) trains a pretrained model on input–output pairs (prompt → response) using teacher-forced cross-entropy loss, teaching it to follow instructions in a specific format.
Supervised fine-tuning (SFT) trains a pretrained model on input–output pairs (prompt → response) using teacher-forced cross-entropy loss, teaching it to follow instructions in a specific format.
Where is it used?
SFT is the first alignment stage for Llama-3-Instruct, Zephyr, and OpenChat; the Datasets library and `trl.SFTTrainer` handle packing and masking of prompt tokens automatically.
How to build it
Format data as `{"prompt": "...", "completion": "..."}`, load with `datasets`, pass to `SFTTrainer(model, train_dataset, args)` from the `trl` library, and train with `trainer.train()`.