The α/r scaling

ProofIntermediate⏱ 30 minFine-Tuningscalingvariancehyperparameters

Problem

Prove $\E\|\frac{\alpha}{r}BA\|_F^2 = \frac{\alpha^2}{r}dk\sigma_B^2\sigma_A^2$ for i.i.d. mean-zero factors.
Show that without the $1/r$ factor the expected squared magnitude grows linearly in $r$ .
If instead $A$ were initialised with variance $\sigma_A^2 \propto 1/r$ , what scaling would make magnitudes rank-invariant, and what does that say about the convention?

Write a complete argument. Every inequality you use should be one you could state precisely.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Adapter parameter counts	Fine-Tuning	Numerical	Foundation	15 min
Derive the attention crossover length	LLM Ops	Proof	Beginner	15 min
The Δ²/12 noise variance	Quantization	Proof	Beginner	15 min
Migrating quantization difficulty	Quantization	Numerical	Intermediate	30 min
Mini-batching and the compute budget	SGD	Numerical	Intermediate	30 min