Implement ridge and find the optimal lambda

CodingIntermediate⏱ 1 hourClassical MLridgecross-validationimplementationconditioning

Problem

Implement ridge regression via the SVD and empirically verify the ridge-beats-OLS theorem.

Plot test MSE against $\lambda$ on a log axis and confirm the minimum is at $\lambda>0$ .
Confirm the optimal $\lambda$ grows with the noise level and with $p/n$ .
Implement leave-one-out cross-validation in closed form using the hat matrix, and check it against brute-force LOOCV.

Implement it, then check the implementation against something you can verify independently.

Solve it here4 tests · Python runs in your browser

⌘/Ctrl + ↵ to run · limit 30s · draft saved in this browser

4 tests will run against your code. The first run downloads the Python runtime — several megabytes, once per browser.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Least squares as orthogonal projection	Classical ML	Proof	Foundation	15 min
Frobenius versus operator norm for random matrices	Random Matrices	Numerical	Intermediate	30 min
Implement backpropagation from scratch run	Deep Learning	Coding	Intermediate	3 hours
Implement kernel ridge regression run	Classical ML	Coding	Intermediate	1 hour
Implement multi-head attention from scratch run	Deep Learning	Coding	Intermediate	3 hours