Skip to content
nonconvexcard

The lasso error vector lies in a cone

ProofGraduate⏱ 1 hourHigh-Dim Statisticslassocone conditionbasic inequalityrestricted eigenvalue

Problem

Let β^\hat\beta minimise 12n∥y−Xβ∥22+λ∥β∥1\frac{1}{2n}\|y-X\beta\|_2^2 + \lambda\|\beta\|_1 with y=Xβ⋆+wy = X\beta^\star+w, and let S=supp⁡(β⋆)S = \operatorname{supp}(\beta^\star), v=β^−β⋆v = \hat\beta-\beta^\star. Prove that on the event 1n∥X⊤w∥∞≤λ/2\frac1n\|X^\top w\|_\infty \le \lambda/2,

∥vSc∥1≤3∥vS∥1.\|v_{S^c}\|_1 \le 3\|v_S\|_1 .

Explain why this is the step that makes a restricted eigenvalue condition — rather than a full eigenvalue condition — sufficient.

Write a complete argument. Every inequality you use should be one you could state precisely.

Hints

Hints — 0 of 3 revealed

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems
SolvedProblemFieldTypeDifficultyTime
Derive the soft-thresholding operatorHigh-Dim StatisticsProofIntermediate30 min
The geometry of l1 versus l2 constraintsHigh-Dim StatisticsProofIntermediate30 min
Implement ISTA and verify the lasso rate runHigh-Dim StatisticsCodingAdvanced3 hours
Union bound over supports for the restricted isometry propertyHigh-Dim StatisticsProofAdvanced1 hour
Open: explicit RIP matrices beyond the square-root barrier abyssHigh-Dim StatisticsProofResearchResearch