Effective bits lost to dynamic range

NumericalBeginner⏱ 15 minQuantizationoutlierseffective bitsgranularity

Problem

A hidden state of 4096 channels has median absolute value $0.5$ and a maximum of $35$ .

Compute the effective bits for typical values under per-tensor 8-bit quantization.
Suppose outliers are confined to 16 known channels. Compute the effective bits for the remaining 4080 channels under per-channel quantization, assuming each has max $\approx 1.5$ .
Compute the memory overhead of per-channel FP16 scales as a fraction of the quantized tensor.

Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Solved	Problem	Field	Type	Difficulty	Time
	Six decibels per bit, and what outliers cost	Quantization	Numerical	Beginner	30 min
	Migrating quantization difficulty	Quantization	Numerical	Intermediate	30 min