LLM Ops & Inference Mathematics

Serving arithmetic: arithmetic intensity and the roofline, KV-cache growth, batching and queueing, and the acceptance algebra of speculative decoding.

3 topics11 problems (4% of the database)assumes LLMs

Level 2 — Advanced

Inference Arithmetic and the Roofline12 min
Why prefill is compute-bound and decode is memory-bound, derived from the FLOP and byte counts of a transformer forward pass.
Forward-pass cost of a transformerThe roofline boundDecode arithmetic intensity is set by the batch size alone
KV Cache, Batching and Little's Law13 min
The memory equation that decides how many sequences fit, and why static batching wastes a logarithmically growing fraction of the GPU.
KV cache sizeMaximum concurrent sequencesLittle's law

Level 3 — Graduate

Speculative Decoding13 min
A draft model proposes, the target model verifies — and a rejection-sampling argument makes the output distribution exactly the target's, with no approximation.
Speculative sampling is exactAcceptance probability equals one minus total variation distanceExpected tokens per verification round

Problems (11)

Open in the browser

Problem	Type	Difficulty	Time
Count the FLOPs of a forward pass	Numerical	Foundation	15 min
Derive the attention crossover length	Proof	Beginner	15 min
KV cache sizing with MHA and GQA	Numerical	Beginner	30 min
Ridge point and the batch size needed to saturate	Numerical	Beginner	15 min
Apply Little's law to a serving target	Proof	Intermediate	30 min
How many users fit on one GPU	Numerical	Intermediate	30 min
Single-stream decode latency floors	Numerical	Intermediate	30 min
Build a serving capacity calculator run	Coding	Advanced	3 hours
Choosing the draft length	Numerical	Advanced	30 min
Exactness of speculative sampling with a modified acceptance rule abyss	Proof	Advanced	1 hour
The cost of waiting for the slowest sequence	Proof	Advanced	1 hour