Dequantization
Dequantization converts quantized integer values back to floating-point for computation. It multiplies the integer by the quantization scale and adds the zero point, reconstructing an approximation of the original float value.
What is Dequantization?
Dequantization converts quantized integer values back to floating-point for computation. It multiplies the integer by the quantization scale and adds the zero point, reconstructing an approximation of the original float value.
Dequantization converts quantized integer values back to floating-point for computation. It multiplies the integer by the quantization scale and adds the zero point, reconstructing an approximation of the original float value.
Where is it used?
During quantized inference (llama.cpp, GPTQ models), weights are stored as INT4/INT8 but dequantized to FP16 on-the-fly for matrix multiplication. Some kernels fuse dequantization with matmul for efficiency. AWQ and GPTQ both dequantize during the forward pass.
How to build it
Given quantized `q` (int8), `scale`, and `zero_point`: `dequantized = scale * (q.float() - zero_point)`. In PyTorch: `torch.dequantize(tensor)` for quantized tensors. For fused inference, use custom CUDA kernels or BitsAndBytes `Linear8bitLt` which dequantizes in the forward pass.