Prove the Gauss–Markov theorem

ProofIntermediate⏱ 30 minClassical MLgauss markovblueunbiased

Theory: Linear, Ridge and Lasso Regression →

Problem

Prove that among linear unbiased estimators $\tilde\beta = Ay$ , least squares has the smallest covariance in the Loewner order. Then state carefully what the theorem does not claim.

Write a complete argument. Every inequality you use should be one you could state precisely.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Least squares as orthogonal projection	Classical ML	Proof	Foundation	15 min
Implement ridge and find the optimal lambda run	Classical ML	Coding	Intermediate	1 hour
Ridge regression in the SVD basis	Classical ML	Proof	Intermediate	30 min
The pass@k estimator	Agents	Proof	Intermediate	30 min
There is always a ridge parameter beating least squares abyss	Classical ML	Proof	Advanced	1 hour