Skip to content
nonconvexcard

Conditional expectation minimises mean squared error

ProofIntermediate⏱ 30 minProbabilityconditional expectationprojectionbayes predictor

Problem

Let X,YX, Y be square-integrable. Prove that

E[(Y−g(X))2] ≥ E[(Y−E[Y∣X])2]\E\big[(Y - g(X))^2\big] \ \ge\ \E\big[(Y - \E[Y\mid X])^2\big]

for every measurable gg with E[g(X)2]<∞\E[g(X)^2]<\infty, with equality iff g(X)=E[Y∣X]g(X)=\E[Y\mid X] almost surely. Conclude that the Bayes-optimal predictor for squared loss is the regression function.

Write a complete argument. Every inequality you use should be one you could state precisely.

Hints

Hints — 0 of 2 revealed

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems
SolvedProblemFieldTypeDifficultyTime
Least squares as orthogonal projectionClassical MLProofFoundation15 min
Characterising orthogonal projectorsFoundationsProofIntermediate30 min
Law of total variance and its useProbabilityProofIntermediate30 min
Moments from tail bounds via the layer-cake formulaProbabilityProofIntermediate30 min
The 1/n variance rate requires uncorrelated samplesProbabilityNumericalIntermediate30 min