Build a serving capacity calculator

CodingAdvanced⏱ 3 hoursLLM Opsrooflinesimulationcapacity planningimplementation

Problem

Implement a capacity planner and validate it against the analysis.

Given model shape, precision and hardware, compute the decode latency floor, the KV cache per token, the maximum batch at a given context, and whether decode is compute- or memory-bound.
Sweep context length and plot achievable batch size and arithmetic intensity together.
Simulate static versus continuous batching with a heavy-tailed length distribution and measure the utilisation gap against the $1/H_B$ prediction.
Add speculative decoding and find the optimal $\gamma$ numerically for measured $\alpha$ .

Implement it, then check the implementation against something you can verify independently.

Solve it here6 tests · Python runs in your browser

⌘/Ctrl + ↵ to run · limit 20s · draft saved in this browser

6 tests will run against your code. The first run downloads the Python runtime — several megabytes, once per browser.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Count the FLOPs of a forward pass	LLM Ops	Numerical	Foundation	15 min
Derive the attention crossover length	LLM Ops	Proof	Beginner	15 min
Implement the cluster sizing formula run	AI Systems	Coding	Beginner	30 min
Ridge point and the batch size needed to saturate	LLM Ops	Numerical	Beginner	15 min
Simulate distance contrast run	Classical ML	Coding	Beginner	30 min