Open: the edge of stability

ProofResearch⏱ ResearchDL TheoryOptimizationopen problemedge of stabilitysharpnesscurvature

Theory: Why Does Deep Learning Optimization Work? →

Problem

Classical analysis says gradient descent on an $L$ -smooth objective needs $\eta < 2/L$ to be stable. In deep network training the largest Hessian eigenvalue is observed to rise until it sits just above $2/\eta$ and then hover there, with the loss decreasing non-monotonically but decreasing.

Explain it: give a class of objectives on which gradient descent provably enters and remains in this regime, and prove the loss still decreases.

This is an open problem. Nobody has a solution, and none is hidden below — what is recorded is the state of the argument. Progress on it would be publishable.

What is known

This problem is open. What follows is an account of what has been proved, what has only been measured, and where the argument stops — not a solution, because there is not one.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Implicit bias of gradient descent on underdetermined least squares	DL Theory	Proof	Intermediate	30 min
What fitting random labels rules out	DL Theory	Proof	Intermediate	30 min
Fit a scaling law and derive the compute-optimal allocation run	DL Theory	Coding	Advanced	3 hours
Sharpness is not a property of the function	DL Theory	Proof	Advanced	30 min
NTK convergence is a PL inequality abyss	DL Theory	Proof	Graduate	1 hour