LoRA Merging
LoRA merging folds the trained adapter ΔW = B·A·(α/r) back into the frozen base weight W' = W + ΔW, producing a single standard model with no runtime overhead from adapter computation.
What is LoRA Merging?
LoRA merging folds the trained adapter ΔW = B·A·(α/r) back into the frozen base weight W' = W + ΔW, producing a single standard model with no runtime overhead from adapter computation.
LoRA merging folds the trained adapter ΔW = B·A·(α/r) back into the frozen base weight W' = W + ΔW, producing a single standard model with no runtime overhead from adapter computation.
Where is it used?
Merging is done via `merge_and_unload()` in `peft` before deployment; `unsloth` and vLLM support merged LoRA serving; multiple LoRA adapters can also be merged with weighted averaging for multi-task models.
How to build it
After training, call `model = model.merge_and_unload()` then `model.save_pretrained("merged")`; load the merged model and verify outputs match the adapter-on-base inference exactly.