The information matrix equality

ProofGraduate⏱ 1 hourFine-Tuningfisherhessianregularitymisspecification

Problem

Prove $F(\theta) = -\E_{p_\theta}[\nabla^2\log p_\theta]$ , stating where each regularity condition is used.
Give an example where the conclusion fails because the support depends on $\theta$ .
Explain what changes when the expectation is taken under a data distribution different from $p_\theta$ , and name the resulting correction.

Write a complete argument. Every inequality you use should be one you could state precisely.

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Problem	Field	Type	Difficulty	Time
Convexity of the logistic loss	Classical ML	Proof	Beginner	15 min
Bounding how far the adapted model moves	Fine-Tuning	Proof	Intermediate	30 min
Derive the EWC objective	Fine-Tuning	Proof	Advanced	1 hour