Size a serving cluster and find the binding constraint

A replica has 80 GB of device memory. The model weights occupy 28 GB in bf16 and workspace takes a further 6 GB. Paged allocation achieves $\eta = 0.92$ of the remainder. A typical request holds $m = 640$ MB of KV cache and takes $W = 5$ s end to end. Offered load is $\lambda = 900$ requests per second and the target utilisation is $\rho^\star = 0.65$.

1. Compute $L_{\max}$ and the replicas required.
2. The hardware has a compute-to-bandwidth ratio of 280 FLOP/byte and the weights are 2 bytes per parameter. Estimate the batch at which decode stops being weight-bound, and say whether this replica is memory-bound or compute-bound.
3. KV quantisation to int8 halves $m$. Recompute the replica count and say what changes about the answer to part 2.
4. Instead of quantising, someone proposes raising $\rho^\star$ to 0.85. Using $W = S/(1-\rho)$ with the same bare service time, say what actually happens.