Convexity of the logistic loss

ProofBeginner⏱ 15 minClassical MLlogisticconvexityhessian

Problem

Show the logistic negative log-likelihood has gradient $X^\top(p-y)$ and Hessian $X^\top SX$ with $S = \operatorname{diag}(p_i(1-p_i))$ , hence is convex. Determine exactly when it is strictly convex.

Write a complete argument. Every inequality you use should be one you could state precisely.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Equality in Jensen's inequality	Foundations	Proof	Beginner	15 min
Every norm is convex, and the operator norm is a supremum of linear functions	Optimization	Proof	Beginner	15 min
First-order characterisation of convexity	Optimization	Proof	Beginner	30 min
In-sample calibration of logistic regression	Classical ML	Proof	Intermediate	30 min
IRLS is a weighted least-squares solve	Classical ML	Proof	Intermediate	30 min