The density of the cosine between random directions

ProofAdvanced⏱ 1 hourRAGcosinedensityspherenear-orthogonality

Theory: Embedding Geometry and Similarity →

Problem

Derive the density $\varrho_d(t)\propto(1-t^2)^{(d-3)/2}$ of $\langle u,v\rangle$ for independent uniform directions in $\R^d$ , compute its variance, and evaluate the standard deviation at $d = 128, 768, 4096$ .

Write a complete argument. Every inequality you use should be one you could state precisely.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Three similarities agree on the unit sphere	RAG	Proof	Foundation	15 min
A sufficient condition for correct retrieval	RAG	Proof	Intermediate	30 min
Measure anisotropy in an embedding space run	RAG	Coding	Intermediate	1 hour
Simulate the measure of a spherical cap run	High-Dim Probability	Coding	Intermediate	30 min
Gaussian width of the sphere and of the sparse set	High-Dim Probability	Proof	Advanced	30 min