Skip to content
nonconvexcard

Prove the Johnson–Lindenstrauss lemma

ProofGraduate⏱ 3 hoursHigh-Dim Probabilityjohnson-lindenstraussunion boundconcentrationflagship

Problem

Prove the Johnson–Lindenstrauss lemma from first principles: for any x1,…,xN∈RDx_1,\dots,x_N\in\R^D and ε∈(0,1)\eps\in(0,1), if A∈Rk×DA\in\R^{k\times D} has i.i.d. N(0,1/k)\Ncal(0,1/k) entries and k≥24ε−2log⁡Nk \ge 24\eps^{-2}\log N, then with probability at least 1−1/N1-1/N,

(1−ε)∥xi−xj∥22≤∥Axi−Axj∥22≤(1+ε)∥xi−xj∥22for all i,j.(1-\eps)\|x_i-x_j\|_2^2 \le \|Ax_i-Ax_j\|_2^2 \le (1+\eps)\|x_i-x_j\|_2^2 \quad\text{for all } i,j .

Your proof must establish the single-vector concentration statement, not merely cite it, and must state clearly where the independence and the fixed-in-advance point set are used.

Write a complete argument. Every inequality you use should be one you could state precisely.

Hints

Hints — 0 of 3 revealed

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems
SolvedProblemFieldTypeDifficultyTime
When is the union bound tight?ProbabilityProofFoundation15 min
Generalisation for a finite classLearning TheoryProofBeginner30 min
Johnson–Lindenstrauss dimension arithmeticHigh-Dim ProbabilityNumericalBeginner15 min
Distance contrast and the failure of nearest neighboursClassical MLProofIntermediate30 min
From an exponential tail bound to almost sure convergenceProbabilityProofIntermediate30 min