Ridge point and the batch size needed to saturate

NumericalBeginner⏱ 15 minLLM Opsrooflineridge pointbatching

Problem

A GPU has peak 990 TFLOP/s in BF16 and 3.35 TB/s of memory bandwidth.

Compute the ridge point in FLOP/byte.
For weight-limited decoding at 2 bytes per parameter, what batch size is needed to reach it?
Repeat for 4-bit weights ( $b = 0.5$ ). Explain the direction of the change.

Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Count the FLOPs of a forward pass	LLM Ops	Numerical	Foundation	15 min
Derive the attention crossover length	LLM Ops	Proof	Beginner	15 min
How many users fit on one GPU	LLM Ops	Numerical	Intermediate	30 min
Single-stream decode latency floors	LLM Ops	Numerical	Intermediate	30 min
Build a serving capacity calculator run	LLM Ops	Coding	Advanced	3 hours