The pass@k estimator

ProofIntermediate⏱ 30 minAgentspass@kunbiasednumerics

Problem

Prove $\widehat{\mathrm{pass}@k} = 1 - \binom{n-c}{k}/\binom{n}{k}$ is the exact probability that a uniform random $k$ -subset of the $n$ samples contains a correct one.
Derive the product form and explain why it is preferred numerically.
Show that estimating pass@k by drawing exactly $k$ samples once is unbiased but has strictly larger variance, and quantify the gap for $n=100$ , $k=1$ , $c=20$ .

Write a complete argument. Every inequality you use should be one you could state precisely.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Sizing best-of-n	Agents	Numerical	Beginner	15 min
Prove the Gauss–Markov theorem	Classical ML	Proof	Intermediate	30 min
The ceiling imposed by a fallible verifier	Agents	Proof	Advanced	30 min
The KL divergence of best-of-n abyss	Agents	Proof	Graduate	1 hour