Activation Quantization
Activation quantization converts the intermediate activation values (outputs of attention, FFN, etc.) to lower precision during the forward pass. This speeds up computation further but requires calibration data to determine activation ranges.
What is Activation Quantization?
Activation quantization converts the intermediate activation values (outputs of attention, FFN, etc.) to lower precision during the forward pass. This speeds up computation further but requires calibration data to determine activation ranges.
Activation quantization converts the intermediate activation values (outputs of attention, FFN, etc.) to lower precision during the forward pass. This speeds up computation further but requires calibration data to determine activation ranges.
Where is it used?
TensorRT and TFLite use activation quantization for maximum inference speed. LLM.int8() quantizes activations but keeps outlier features in FP16. Full INT8 inference (weights + activations) is used in production serving where throughput matters more than setup complexity.
How to build it
Calibrate: run ~100 representative inputs through the model and record activation min/max per layer. Compute scales: `act_scale = act.abs().max() / 127`. In forward: `q_act = torch.round(act / act_scale).clamp(-128, 127).to(torch.int8); act = q_act.float() * act_scale`. Use PyTorch `torch.quantization` API or TensorRT PTQ.