The true bit width of a group-quantized format

NumericalFoundation⏱ 15 minQuantizationgroup quantizationoverheadhonesty

Theory: Uniform Quantization and Its Error →

Problem

A "4-bit" weight format stores one FP16 scale and one FP16 zero point per group of $g$ weights. Compute the true average bits per weight for $g = 32, 64, 128$ , and for a variant storing only a scale. Comment on what this means for comparing published compression ratios.

Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
The round-trip error bound	Quantization	Proof	Foundation	15 min
Six decibels per bit, and what outliers cost	Quantization	Numerical	Beginner	30 min
The Δ²/12 noise variance	Quantization	Proof	Beginner	15 min
Optimal clipping for a Laplace-distributed tensor	Quantization	Proof	Advanced	1 hour