Choosing the draft length
NumericalAdvanced⏱ 30 minLLM Opsspeculative decodingoptimisationspeedup
Problem
With acceptance rate and relative draft cost , speedup is .
- For , , tabulate for and find the optimum.
- Show that as the optimal diverges, and give the limiting speedup for fixed .
- A team reports and claims a speedup. Is that consistent?
Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.
Solution
Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.
Related problems
All problems| Solved | Problem | Field | Type | Difficulty | Time |
|---|---|---|---|---|---|
| Optimal checkpointing frequency | Fine-Tuning | Proof | 30 min | ||
| Exactness of speculative sampling with a modified acceptance rule abyss | LLM Ops | Proof | 1 hour | ||
| Implement gradient descent and verify all four rates run | Optimization | Coding | 3 hours | ||
| Optimal clipping for a Laplace-distributed tensor | Quantization | Proof | 1 hour |