Reading the Robbins–Monro conditions off the proof
ProofIntermediate⏱ 30 minSGDrobbins-monrostep sizeschedule
Problem
From the recursion , explain why the conditions
are exactly what is needed for . Check which of , , satisfy them, and what each gives.
Write a complete argument. Every inequality you use should be one you could state precisely.
Solution
Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.
Related problems
All problems| Solved | Problem | Field | Type | Difficulty | Time |
|---|---|---|---|---|---|
| The exact step-size threshold on a quadratic | Optimization | Proof | 15 min | ||
| Mini-batching and the compute budget | SGD | Numerical | 30 min | ||
| SGD on convex Lipschitz objectives | SGD | Proof | 30 min | ||
| Expected error does not give a high-probability guarantee | SGD | Proof | 30 min | ||
| Implement SGD and locate the noise floor run | SGD | Coding | 3 hours |