Why Does Deep Learning Optimization Work?
What is actually proven about training deep networks — and, stated just as carefully, what is not.
Level 4 — Research
Problems (7)
Open in the browser| Solved | Problem | Type | Difficulty | Time |
|---|---|---|---|---|
| Implicit bias of gradient descent on underdetermined least squares | Proof | 30 min | ||
| What fitting random labels rules out | Proof | 30 min | ||
| Fit a scaling law and derive the compute-optimal allocation run | Coding | 3 hours | ||
| Sharpness is not a property of the function | Proof | 30 min | ||
| NTK convergence is a PL inequality abyss | Proof | 1 hour | ||
| Open: does SGD find global minima of deep networks? abyss | Proof | Research | ||
| Open: the edge of stability abyss | Proof | Research |