Mathematical Foundations
Linear algebra, matrix calculus and the analysis prerequisites every later section silently assumes.
3 topics12 problems (4% of the database)
Level 0 — Foundations
- Linear Algebra for Machine Learning16 min
Inner products, orthogonal projection, the spectral theorem, the SVD and the matrix norms that every later bound is written in.
Cauchy–Schwarz inequalityOrthogonal projection onto a subspaceSpectral theorem for real symmetric matrices - Multivariable and Matrix Calculus13 min
Gradients, Jacobians, Hessians and Taylor expansion, set up so that backpropagation and every smoothness assumption later are routine.
Taylor's theorem with integral remainderDescent lemma for L-smooth functionsChain rule for compositions - Analysis Prerequisites6 min
Suprema, limits, Lipschitz continuity, compactness and the two or three analysis facts that the probability and optimisation proofs quietly rely on.
Jensen's inequalityHölder's inequalityMonotone and dominated convergence
Problems (12)
Open in the browser| Solved | Problem | Type | Difficulty | Time |
|---|---|---|---|---|
| Equality in Cauchy–Schwarz | Proof | 15 min | ||
| Equality in Jensen's inequality | Proof | 15 min | ||
| Lipschitz constants depend on the norm | Numerical | 15 min | ||
| Tightness of the descent lemma | Proof | 15 min | ||
| All directional derivatives exist, yet the function is not differentiable | Proof | 30 min | ||
| Bounding the spectral norm by row and column sums | Proof | 30 min | ||
| Characterising orthogonal projectors | Proof | 30 min | ||
| Implement power iteration and verify the convergence rate run | Coding | 1 hour | ||
| The softmax Jacobian is a covariance | Proof | 30 min | ||
| The supremum is 1-Lipschitz in the uniform norm | Proof | 15 min | ||
| Gradient and Hessian of log-determinant | Proof | 30 min | ||
| Schur complement and block positive semidefiniteness | Proof | 30 min |