Deep Dive

Production RAG from the Retrieval Up

How retrieval-augmented generation works when it works, and exactly where it fails when it does not. Chunking, embeddings, lexical and hybrid retrieval, reranking, measurement, context assembly, freshness and access control, tracing, and injection through retrieved text: the mechanism of each layer, with the numbers, so you can tune a system rather than swap components at random.

A chain of contractsparsechunkembedretrievererankassembleeach layer owes the next something specific↑ chunking: where meaning is lost firstthe generatorlabelled set~400 q → passageone set measures every layer · recall@k per slicewrong answer, right document in the index?force the gold passage into the prompt:if the answer flips, the breach is upstream
If you only have an hour

Read 00 for the pipeline and the one eval set everything else is measured on, then 04 (why dense alone misses) and 09 (how to find which layer is at fault). Those three change how you look at every RAG system you meet.

Study pathsPick the plan that fits the time you have
A week of evenings

Two chapters a night in order. Build the labelled set from 06 on your own corpus as you go; every later chapter gives you something to measure with it.

One afternoon

00, 02, 04, 05, 07, 09. Read the Takeaway of the rest.

Before an interview

00, 04, 06, 09 here; then the interview framing in AI Engineer Interview Prep 01–03.

Section A · Orient
00 Start Here The pipeline as a chain of contracts, the one labelled set every chapter measures against, and the running example.
Section B · Building the index
01 The Retrieval Contract What the generator needs from retrieval, stated precisely: enough of the right passages, in a usable shape, with provenance, inside a token budget. Every later design choice is judged against this. 02 Parsing and Chunking Where documents lose their meaning before an embedding ever sees them: PDF extraction, tables, headings as context, fixed windows against structure-aware and parent-child chunking, overlap, and how to measure a chunking choice. 03 Embeddings What an embedding model actually encodes, why domain vocabulary and identifiers defeat it, dimension and quantisation trade-offs, query and document asymmetry, and how to choose and evaluate a model for your corpus.
Section C · Retrieving
04 Lexical, Dense and Hybrid Retrieval BM25 and dense retrieval fail on different queries. Why, with examples; Reciprocal Rank Fusion and weighted fusion; filters and metadata; the ANN index choices behind the vector store and what they cost in recall. 05 Reranking Cross-encoders, late interaction and LLM rerankers: what each computes that a bi-encoder cannot, the latency and cost per hundred candidates, and when the first stage is already good enough. 06 Measuring Retrieval Building a labelled query-to-passage set from production traffic, recall@k, MRR and nDCG and what each hides, slicing by query type, and how to run a sweep so a change is a measurement and not an opinion.
Section D · Generating and operating
07 Context Assembly and Grounding Choosing k, ordering and lost-in-the-middle, deduplication, token budgets, citation formats the model can honour, abstention when the passages do not answer, and measuring faithfulness. 08 Freshness, Tenancy and Access Control Keeping the index true to the source: change detection, re-embedding, deletes and versions, multi-tenant isolation, document-level permissions enforced at retrieval time rather than in the prompt. 09 Debugging a RAG System The trace you need per request, the bisection that separates retrieval faults from generation faults, the six failure classes and the test that identifies each, and the dashboard that catches regressions before users do. 10 Injection Through Retrieved Text Retrieved documents are untrusted input. How indirect prompt injection and data exfiltration work through RAG, the defences that hold and the ones that do not, and how to test them.