Derive the soft-thresholding operator

ProofIntermediate⏱ 30 minHigh-Dim Statisticsproximalsoft thresholdl1subdifferential

Theory: Sparse Recovery, Compressed Sensing and the Lasso →

Problem

Solve

\min_{z\in\R}\ \tfrac{1}{2}(z-v)^2 + \eta|z|

in closed form, using the subdifferential optimality condition $0\in\partial f(z)$ . Explain why the solution is exactly zero for a range of $v$ , and contrast with the ridge case $\min_z\tfrac12(z-v)^2+\tfrac\eta2z^2$ .

Write a complete argument. Every inequality you use should be one you could state precisely.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
The proximal operator of the l1 norm	Advanced Optimization	Proof	Beginner	15 min
The geometry of l1 versus l2 constraints	High-Dim Statistics	Proof	Intermediate	30 min
Implement ISTA and verify the lasso rate run	High-Dim Statistics	Coding	Advanced	3 hours
Union bound over supports for the restricted isometry property	High-Dim Statistics	Proof	Advanced	1 hour
The lasso error vector lies in a cone abyss	High-Dim Statistics	Proof	Graduate	1 hour