End-to-end recall arithmetic for a RAG system

NumericalBeginner⏱ 15 minRAGrecallpipelinebudgetdiagnosis

Theory: Retrieval as a Latent-Variable Model →

Problem

A team reports $70\%$ answer accuracy and proposes to fine-tune the generator. The pipeline has chunk recall $0.93$ , index recall $0.88$ , top- $5$ truncation recall $0.92$ . Compute the retrieval ceiling, and advise.

Compute the requested quantities and say what they mean. A number without an interpretation is not an answer.

Solution

Opening this before you have a real attempt on paper is the fastest way to feel like you understand something you cannot reproduce.

Related problems

All problems

Problem	Field	Type	Difficulty	Time
Recall composes multiplicatively	RAG	Numerical	Beginner	15 min
RAG-sequence and RAG-token are different models	RAG	Proof	Intermediate	30 min
Build and instrument a minimal RAG pipeline run	RAG	Coding	Advanced	1 day
Implement SimHash LSH and measure recall run	RAG	Coding	Advanced	3 hours
The RAG evidence lower bound	RAG	Proof	Advanced	30 min