Derive the EWC objective
ProofAdvanced⏱ 1 hourFine-Tuningewclaplacebayes
Problem
Derive the elastic weight consolidation objective from a Laplace approximation, and then identify precisely which step fails for a pretrained neural network. Propose a modification that does not rely on the failing step, and say what it costs.
Write a complete argument. Every inequality you use should be one you could state precisely.
Solution
Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.
Related problems
All problems| Solved | Problem | Field | Type | Difficulty | Time |
|---|---|---|---|---|---|
| Bounding how far the adapted model moves | Fine-Tuning | Proof | 30 min | ||
| Optimal clipping for a Laplace-distributed tensor | Quantization | Proof | 1 hour | ||
| The information matrix equality abyss | Fine-Tuning | Proof | 1 hour |