Forgetting and Regularisation

Where the quadratic penalty in elastic weight consolidation comes from, why the Fisher information is the right curvature, and what the derivation assumes that deep networks violate.

Fine-TuningGraduate⏱ 1 hour11 min readcatastrophic forgettingewcfisher informationlaplace approximationregularisation

3 problems →

AssumesLow-Rank Adaptation · Expectation, Variance and Conditional Expectation · Empirical Risk Minimisation and Uniform Convergence

Fisher information is curvature

TheoremInformation matrix equality

Let $\lbrace p_\theta \rbrace$ be a family of densities, smooth in $\theta$ , with support not depending on $\theta$ and with differentiation under the integral sign permitted twice. Define the Fisher information

F(\theta) = \E_{x\sim p_\theta}\!\left[\nabla_\theta \log p_\theta(x)\,\nabla_\theta\log p_\theta(x)^\top\right].

Then

F(\theta) = -\,\E_{x \sim p_\theta}\!\left[\nabla^2_\theta \log p_\theta(x)\right].

Assumptions

The regularity conditions are doing real work: fixed support, and the exchange of $\nabla$ with $\int$ . Note also that both expectations are under the model $p_\theta$ , not under the data distribution. When the model is misspecified the two sides differ, and the discrepancy is exactly what the sandwich (robust) covariance corrects for.

Key idea

The Fisher is the expected curvature of the negative log-likelihood. So $F_{ii}$ large means the old task's loss rises steeply when weight $i$ moves — the weight is load-bearing. $F_{ii}\approx 0$ means the loss is flat in that direction and the weight can be repurposed at no cost. It is also positive semidefinite by construction, being an expectation of outer products, which is what makes the penalty below a sensible convex pull.

From Bayes to a quadratic penalty

PropositionThe elastic-weight-consolidation objective

Let $\theta^\star_A$ be a mode of the posterior $p(\theta \mid \mathcal D_A)$ . Approximating that posterior by a Gaussian via a second-order expansion at $\theta^\star_A$ , the maximum a posteriori objective for the combined data $\mathcal D_A \cup \mathcal D_B$ becomes, up to additive constants,

\min_{\theta}\; \mathcal L_B(\theta) \;+\; \frac{\lambda}{2}\big(\theta-\theta^\star_A\big)^\top F\big(\theta-\theta^\star_A\big),

where $\mathcal L_B$ is the negative log-likelihood of $\mathcal D_B$ and $F$ the Fisher information at $\theta^\star_A$ .

Derivation

$\log p(\theta \mid \mathcal D_A, \mathcal D_B) = \log p(\mathcal D_B\mid\theta) + \log p(\theta\mid\mathcal D_A) + \text{const}.$
Conditional independence of the two datasets given the parameters.
$\log p(\theta\mid\mathcal D_A) \approx \log p(\theta^\star_A\mid\mathcal D_A) - \tfrac12(\theta-\theta^\star_A)^\top H_A (\theta-\theta^\star_A),$
with $H_A = -\nabla^2 \log p(\theta\mid\mathcal D_A)\big|_{\theta^\star_A} \succeq 0$ .
Second-order Taylor expansion of the old-task log posterior at its mode; the linear term vanishes there.
For a flat prior and $n_A$ observations, $H_A \approx n_A F(\theta^\star_A)$ by the theorem above. Absorbing $n_A$ and the prior strength into a single hyperparameter $\lambda$ gives $H_A = \lambda F$ .
Identify the curvature with the Fisher using the information matrix equality.
$\min_\theta\ \mathcal L_B(\theta) + \frac{\lambda}{2}(\theta-\theta^\star_A)^\top F (\theta-\theta^\star_A). \qquad\blacksquare$
Negate to turn maximisation into minimisation.

In practice $F$ is replaced by its diagonal, giving the familiar per-weight form

\mathcal L_B(\theta) + \frac{\lambda}{2}\sum_i F_{ii}\big(\theta_i - \theta^\star_{A,i}\big)^2 .

Common mistake

Three approximations were made and all three are questionable for deep networks:

The quadratic expansion is local. Fine-tuning moves weights far enough that a second-order model of the old loss is not reliable, and the expansion has no error control here.
$\theta^\star_A$ is not a mode. The derivation expands at a posterior mode. A pretrained network sits at some point found by SGD, which is not a stationary point of the objective in any verified sense, so the vanishing linear term is assumed rather than established.
Diagonal Fisher discards the couplings. The off-diagonal entries encode exactly the correlated directions that matter most, and dropping them is done for tractability — $F$ is $P \times P$ — not for any reason of accuracy.

EWC is a principled heuristic: the Bayesian story motivates the form of the penalty, and the penalty's usefulness on deep networks is an empirical finding, not a consequence of the derivation.

What low rank buys

PropositionLow rank bounds how far the function can move

Let a layer compute $x \mapsto W_0 x$ and be adapted to $x\mapsto (W_0 + \tfrac\alpha r BA)x$ . Then for every input $x$ ,

\big\|(W_0 + \tfrac{\alpha}{r}BA)x - W_0x\big\|_2 \ \le\ \frac{\alpha}{r}\,\|B\|_2\,\|A\|_2\,\|x\|_2 ,

so weight decay on $A$ and $B$ directly limits the layer's deviation from the pretrained function.

This is a crude bound — it is tight only when $x$ aligns with the top right singular vector of $BA$ — but it makes a structural point. Constraining the update to rank $r$ and penalising the factors gives an explicit, tunable budget on how far the adapted model may move from the pretrained one, and recovers the pretrained function exactly at $B=0$ . Full fine-tuning has no comparable knob: its deviation is limited only by the optimiser's trajectory.

This is one reason low-rank adaptation is often reported to forget less than full fine-tuning at matched downstream performance. It is a plausible mechanism supported by the inequality, not a proof that it does — the bound constrains one layer's output, and forgetting is a property of the composed network on a distribution the bound says nothing about.

Empirical observationNarrow fine-tuning degrades unrelated capability

Empirical observation — a regularity seen in experiments. No theorem on this page establishes it in general.

Fine-tuning a general model on a narrow distribution reliably reduces measured performance on capabilities the base model previously had, with degradation increasing in the number of steps and the learning rate, and decreasing when a fraction of general data is mixed into the fine-tuning set.

Open problem

Open problem — no proof and no counterexample is known to the author of this page.

There is no quantitative theory predicting how much a given fine-tuning run will forget. Inputs to such a theory would include the divergence between the fine-tuning and pretraining distributions, the optimisation trajectory, and the geometry of the pretrained solution — and no current account combines them into a prediction that can be checked before training. Practitioners run the evaluation suite before and after, which is the honest method available.

References

PaperJ. Kirkpatrick et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks
ChapterM. McCloskey, N. Cohen (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning ProblemThe original identification of the phenomenon, three decades before it mattered commercially.

These works are cited as sources to read, not reproduced. All exposition, examples and problems on this site are written independently; where an argument is standard, the citation points to a canonical treatment.

Results stated on this page

TheoremInformation matrix equalityproof includedUnder regularity, the Fisher information equals the expected Hessian of the negative log-likelihood, so the Fisher is curvature.
PropositionThe elastic-weight-consolidation objectiveproof includedA Laplace approximation to the posterior from the first task turns sequential learning into the new task's loss plus a Fisher-weighted quadratic pull toward the old weights.
PropositionLow rank bounds how far the function can moveproof includedA rank-r adapter changes any layer's output by at most the adapter's spectral norm times the input norm, giving an explicit and controllable deviation budget.
Empirical observationNarrow fine-tuning degrades unrelated capabilityFine-tuning on a narrow distribution measurably reduces performance on tasks the base model previously handled, with the loss growing with steps and learning rate.

Problems (3)

All problems