RAG-sequence and RAG-token are different models
ProofIntermediate⏱ 30 minRAGragmarginalisationmixtureattribution
Problem
Show that a mixture of products differs from a product of mixtures by an explicit example, and explain the consequence for multi-hop questions and for attribution.
Write a complete argument. Every inequality you use should be one you could state precisely.
Solution
Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.
Related problems
All problems| Solved | Problem | Field | Type | Difficulty | Time |
|---|---|---|---|---|---|
| End-to-end recall arithmetic for a RAG system | RAG | Numerical | 15 min | ||
| Build and instrument a minimal RAG pipeline run | RAG | Coding | 1 day | ||
| The RAG evidence lower bound | RAG | Proof | 30 min | ||
| Retrieval error propagates into generation abyss | RAG | Proof | 1 hour |