Zero Point
The zero point is the integer value that corresponds to the floating-point value 0.0 in asymmetric quantization. It shifts the integer range so that the float zero maps to an exact integer, important because activations are often non-symmetric around zero.
What is Zero Point?
The zero point is the integer value that corresponds to the floating-point value 0.0 in asymmetric quantization. It shifts the integer range so that the float zero maps to an exact integer, important because activations are often non-symmetric around zero.
The zero point is the integer value that corresponds to the floating-point value 0.0 in asymmetric quantization. It shifts the integer range so that the float zero maps to an exact integer, important because activations are often non-symmetric around zero.
Where is it used?
Asymmetric quantization with zero points is used in TFLite for mobile inference and in some LLM quantization schemes. ReLU activations, which are non-negative, benefit from asymmetric quantization. Symmetric quantization (zero_point=0) is more common for LLM weights.
How to build it
Compute: `scale = (x.max() - x.min()) / 255; zero_point = round(-x.min() / scale)`. Quantize: `q = torch.round(x / scale + zero_point).to(torch.uint8)`. Dequantize: `deq = scale * (q.float() - zero_point)`. Verify: `deq` approximates `x`.