Production RAG & GraphRAG

Retrieval-augmented generation separates a model’s parametric behavior from a corpus that can be searched at request time. The retriever proposes evidence; the generator uses a bounded context to produce an answer. GraphRAG adds structured entities, relationships, claims, communities, and summaries so retrieval can follow connections and answer questions that ordinary passage similarity handles poorly. It is an evidence architecture, not a promise that putting text in a graph makes generation factual.

Start by classifying queries. A local query asks about a named or discoverable entity and its neighborhood: “What obligations apply to Project Atlas?” A global query asks for themes or patterns across the corpus: “What operational risks recur across all incident reports?” A path query asks how two things are connected. A temporal query asks what changed or what was known at a time. An aggregation query requests counts or distributions. Each class needs different index products, retrieval plans, evidence shapes, and evaluation sets.

Vanilla dense retrieval works well when a relevant passage is semantically close to the query and mostly self-contained. Graph retrieval earns its cost when evidence is distributed across documents, references use aliases, intermediate relationships matter, corpus-wide synthesis is required, or a user needs an inspectable connection. It can perform worse when extraction is noisy, relationships are ornamental, latency is strict, or the corpus changes too quickly for graph maintenance. Maintain a lexical and dense baseline so graph complexity must demonstrate incremental value.

Define the boundary between source truth and generated index artifacts. Documents and text units are evidence-bearing source projections. Extracted entities, relationships, claims, community assignments, and reports are generated artifacts whose errors must remain traceable to source spans and pipeline versions. A community summary is useful retrieval material, but it is not an authority independent of the documents it summarizes. Generated text should never silently become a new primary source on the next indexing run.

A request travels through explicit stages: classify the query; resolve entities or themes; select local, global, hybrid, or adaptive search; retrieve candidates under an authorization scope; assemble a token-budgeted context; generate with citation constraints; verify support; and return or abstain. Capture the plan and artifacts. If the answer is wrong, operators need to determine whether the failure came from ingestion, extraction, resolution, community detection, retrieval, context packing, generation, or policy.

GraphRAG contains at least three graphs. The evidence graph connects source units and their provenance. The semantic graph connects normalized entities and typed claims. The operational graph records releases, models, prompts, jobs, evaluations, and access policy. Conflating them causes generated relationships to look authoritative and makes rollback difficult. The serving layer may denormalize them, but identifiers and roles must survive.

Define a query contract before choosing algorithms. It states intended user, authorized corpus, freshness objective, expected evidence granularity, maximum latency and cost, acceptable abstention, and required answer structure. Global sensemaking may accept seconds and multiple model calls; an interactive entity lookup may not. A high-stakes compliance answer may require exact citations and conservative abstention; brainstorming may optimize breadth.

Threat boundaries are present from the start. Documents are untrusted input, even when internal. Extracted instructions are data, not commands. Authorization must filter evidence before summaries and rankings can influence the answer. Tenant or classification boundaries that are applied only after global community construction can leak themes through reports. Build indexes inside compatible security scopes or use a proven policy-aware aggregation design.

Cost has an offline and online component. Indexing invokes parsers, extraction models, resolution, graph algorithms, summaries, and embeddings. Querying invokes searches, traversals, reranking, context construction, and generation. Track cost and quality by query class rather than reporting one average. A system is production-ready only when it can explain why graph retrieval is selected, how its evidence is bounded, and what happens when an index component is unavailable.