Modern AI / LLM Mathematics
Autoregressive factorisation, cross-entropy, attention at scale, preference optimisation and scaling laws.
Level 3 — Graduate
Level 4 — Research
- RLHF, Bradley–Terry and DPO12 min
The preference model, the KL-regularised RL objective, its closed-form optimum, and the algebra that turns it into a supervised loss.
Bradley–Terry preference modelThe reward is identifiable only up to a prompt-dependent shiftClosed form of the KL-regularised RL optimum - Scaling Laws, Fine-Tuning and In-Context Learning9 min
The compute-optimal allocation computation, LoRA's rank argument, and what is actually known about how in-context learning works.
Compute-optimal allocation under a power-law lossThe 6ND FLOP estimateLoRA parameter count and the rank argument
Problems (14)
Open in the browser| Solved | Problem | Type | Difficulty | Time |
|---|---|---|---|---|
| LoRA parameter accounting | Numerical | 15 min | ||
| Maximum likelihood is KL minimisation | Proof | 15 min | ||
| Perplexity is not comparable across tokenisers | Numerical | 15 min | ||
| The 6ND FLOP estimate | Numerical | 15 min | ||
| The cross-entropy gradient is the prediction error | Proof | 15 min | ||
| Derive the compute-optimal allocation | Proof | 30 min | ||
| Mixture-of-experts sparsity accounting | Numerical | 30 min | ||
| Rewards are identifiable only up to a prompt-dependent shift | Proof | 30 min | ||
| The irreducible loss is the conditional entropy | Proof | 30 min | ||
| Temperature monotonically controls entropy | Proof | 30 min | ||
| The DPO gradient is a weighted contrastive step | Proof | 30 min | ||
| Closed form of the KL-regularised RL optimum abyss | Proof | 1 hour |