Retrieval Augmented Generation for grounding AI.
For a small example, one claim cites allowed doc-1, one has no citation, and one cites allowed doc-2 plus unknown doc-9. Resolve citation handles against the selected evidence, count two of three claims as covered, then check semantic support separately.
Product documentation changes every week and some passages are permissioned. Retrieve candidates, rerank, filter permissions before Top-K selection, generate claims, then report retrieval recall separately from citation coverage and semantic support.
Initializing Interactive Playground...
Before running the pipeline, predict how dense versus reranked Top-2 selection changes recall, then identify which of the three generated claims preserve an allowed evidence connection.
The starter reports zero coverage for every answer, so the system cannot distinguish grounded claims from uncited or unknown evidence references.
Check: The result is 0.667, the empty case is safe, and your explanation distinguishes citation-handle coverage from semantic claim support.
Failing starter lab
Measure the fraction of claims connected to at least one allowed evidence id. This validates citation handles, not semantic support.
Run the incomplete starter before opening hints. Use the first wrong output or failed check to form a repair hypothesis.
Explanations after each answer · 80% to pass · RAG
Retrieval Systems book
10 chapters · 4-6 hours chapter study · 2-4 hours labs · 5-8 hours project
Chapter 1 of 10 · Mental model
Retrieval-augmented generation, or RAG, is an architecture in which a system retrieves external evidence and conditions generation on that evidence. The original RAG formulation combined a pretrained generator’s parametric memory with retrieved passages from a non-parametric memory. In production, the important shift is not merely giving a model more text. It is creating a verifiable contract among a question, an authorized corpus snapshot, retrieved evidence, generated claims, citations, and an abstention policy.
Separate three kinds of memory. Parametric memory is encoded in model weights and is difficult to inspect or update precisely. Retrieval memory is the versioned corpus and indexes that can be changed without retraining the generator. Run memory is the temporary state of a particular workflow: query transformations, candidate lists, selected context, tool results, and prior turns. Confusing them causes failures. A conversational statement is not automatically authoritative corpus evidence; a model recollection is not a citation; a retrieved passage is not automatically trustworthy.
A minimal RAG system has six stages. Ingestion creates stable, policy-bearing evidence units. Query processing resolves intent, scope, and authorized filters. Retrieval produces candidates with provenance. Context construction chooses a bounded, coherent evidence packet. Generation produces claims under an explicit evidence-use instruction and output schema. Verification evaluates support, citation validity, safety, and answer policy. More advanced systems loop among these stages, but the responsibilities remain distinct.
Grounding is a relation between claims and evidence. An answer can be factually true yet unsupported by the supplied sources, or faithfully restate a poisoned source and still harm the user. Therefore distinguish correctness, faithfulness, source authority, relevance, completeness, and policy compliance. Faithfulness asks whether claims follow from evidence. Correctness asks whether claims match the world or a trusted reference. Authority asks whether the evidence is acceptable for this decision. A system needs all relevant dimensions rather than calling them collectively “hallucination.”
RAG cannot guarantee truth. Retrieval may miss decisive evidence, return obsolete content, or surface malicious text. The generator may ignore evidence, merge incompatible versions, or cite a passage that does not entail its claim. Citations can be cosmetically plausible but wrong. The architecture should make these failures observable and choose safe behavior: ask a clarifying question, state uncertainty, narrow scope, retrieve again, or abstain.
Define the answer contract before prompts. It should state allowed sources, temporal and jurisdiction scope, citation granularity, treatment of conflicts, minimum support, response schema, forbidden actions, and abstention language. For a customer-support assistant, community posts may be useful leads but official versioned documentation may be required for definitive instructions. For a medical or legal workflow, retrieval may assist a qualified reviewer rather than produce autonomous advice.
Evidence identity flows end to end. Every context unit carries passage ID, source revision, heading, effective dates, access decision, retrieval scores, and locator. The generator can cite short opaque evidence handles rather than fabricate URLs. A post-processor resolves handles only if they were actually supplied. Claim-level verification maps each material statement to one or more handles. This does not prove truth, but it prevents citations to nonexistent context and supports audit.
At toy scale, a three-document animal corpus demonstrates retrieval and citations. The system should answer only from supplied facts and say it lacks evidence for an absent fact. At application scale, an internal policy assistant retrieves the employee’s jurisdiction and policy version, presents quoted evidence, cites section IDs, and routes conflicting or high-consequence cases to HR. At system scale, a multi-tenant research agent searches several governed corpora, executes adaptive retrieval under a budget, preserves lineage across transformations, and produces an evidence graph for every conclusion.
Measure the chain, not only answer fluency. Corpus coverage, retrieval recall, context precision, context coverage, claim support, citation precision, answer correctness, abstention quality, latency, cost, freshness, and security all matter. An excellent generator cannot use evidence it never receives. Perfect retrieval does not force faithful synthesis. Aggregate scores must be paired with slices and traces.
This book progresses from corpus provenance through query and retrieval, context engineering, grounded generation, adaptive workflows, first-principles implementation, evaluation, security, and production operations. A dependable RAG system is not a prompt attached to a vector database. It is an evidence pipeline whose claims can be inspected, challenged, reproduced, and safely declined.
Example 1 · Toy
Answer questions from three short passages and decline unsupported facts.
Work through it
Retrieve by term overlap, build evidence handles, generate only supported statements, and reject a citation not present in context.
Evidence to inspect
Example 2 · Application
An employee asks about leave rules that vary by region and effective date.
Work through it
Resolve identity and jurisdiction, retrieve current authoritative sections, show conflicts, cite exact revisions, and escalate consequential ambiguity.
Evidence to inspect
Example 3 · System
A workflow synthesizes evidence across private and public corpora.
Work through it
Authorize each source, run bounded query branches, deduplicate and rank evidence, record an evidence graph, verify claims, and retain a replayable trace.
Evidence to inspect
Ideas to understand
What Is Retrieval-Augmented Generation (RAG)?
IBM Technology
Understand the retrieval, context, generation, and evaluation stages of a grounded-answer system.
Primary sources and free technical reading for RAG.