Why exactly one factor starts at zero

ProofIntermediate⏱ 30 minFine-Tuninginitialisationgradientstationary point

Problem

For $W = W_0 + \frac{\alpha}{r}BA$ :

Derive $\partial\mathcal L/\partial A$ and $\partial\mathcal L/\partial B$ in terms of $G = \partial\mathcal L/\partial W$ .
Show $A = B = 0$ is a stationary point of the adapter parameters, and that it is not a local minimum in general.
Show that $B=0$ with $A$ random is not stationary, and that the first update makes both gradients non-zero.

Write a complete argument. Every inequality you use should be one you could state precisely.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Adapter parameter counts	Fine-Tuning	Numerical	Foundation	15 min
The cross-entropy gradient is the prediction error	LLMs	Proof	Beginner	15 min
Gradient magnitude through a deep chain	Deep Learning	Proof	Intermediate	30 min
Non-convex gradient descent reaches an approximate stationary point	Optimization	Proof	Intermediate	30 min
The softmax Jacobian as a variance, and saturation	Deep Learning	Proof	Intermediate	30 min