Conditional expectation minimises mean squared error
ProofIntermediate⏱ 30 minProbabilityconditional expectationprojectionbayes predictor
Problem
Let be square-integrable. Prove that
for every measurable with , with equality iff almost surely. Conclude that the Bayes-optimal predictor for squared loss is the regression function.
Write a complete argument. Every inequality you use should be one you could state precisely.
Hints
Hints — 0 of 2 revealed
Solution
Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.
Related problems
All problems| Solved | Problem | Field | Type | Difficulty | Time |
|---|---|---|---|---|---|
| Least squares as orthogonal projection | Classical ML | Proof | 15 min | ||
| Characterising orthogonal projectors | Foundations | Proof | 30 min | ||
| Law of total variance and its use | Probability | Proof | 30 min | ||
| Moments from tail bounds via the layer-cake formula | Probability | Proof | 30 min | ||
| The 1/n variance rate requires uncorrelated samples | Probability | Numerical | 30 min |