RAG Mathematics
Embeddings, similarity geometry, approximate nearest-neighbour search, and retrieval as a latent-variable model.
Level 3 — Graduate
- Embedding Geometry and Similarity11 min
Cosine, dot product and Euclidean distance and the exact relations between them; why high-dimensional near-orthogonality is what makes retrieval possible; and the maximum-of-N problem that limits it.
Cosine, dot product and Euclidean distance agree on normalised vectorsWithout normalisation the norm dominates the rankingDistribution of the cosine between random directions - Vector Search — Exact and Approximate Nearest Neighbours11 min
The complexity of exact search, why tree methods fail in high dimension, and the mathematics of LSH, IVF, product quantisation and HNSW.
Cost of exact nearest-neighbour searchWhy space-partitioning trees fail in high dimensionLocality-sensitive hash family
Level 4 — Research
Problems (16)
Open in the browser| Solved | Problem | Type | Difficulty | Time |
|---|---|---|---|---|
| Three similarities agree on the unit sphere | Proof | 15 min | ||
| End-to-end recall arithmetic for a RAG system | Numerical | 15 min | ||
| Product quantisation budget arithmetic | Numerical | 30 min | ||
| Recall composes multiplicatively | Numerical | 15 min | ||
| A sufficient condition for correct retrieval | Proof | 30 min | ||
| Measure anisotropy in an embedding space run | Coding | 1 hour | ||
| RAG-sequence and RAG-token are different models | Proof | 30 min | ||
| The random-hyperplane collision probability | Proof | 30 min | ||
| Build and instrument a minimal RAG pipeline run | Coding | 1 day | ||
| Implement SimHash LSH and measure recall run | Coding | 3 hours | ||
| The density of the cosine between random directions | Proof | 1 hour | ||
| The maximum of N distractor similarities | Proof | 30 min |