Adaptive and corrective RAG workflows

A fixed RAG pipeline retrieves once and generates once. That is often the safest starting point because its cost and evidence path are simple. Adaptive workflows add decisions: whether retrieval is needed, which corpus to query, whether evidence is adequate, how to rewrite a query, whether to search again, and when to stop. These decisions can improve difficult cases, but every loop increases latency, cost, state, and attack surface.

Model the workflow as a bounded state machine. States might include scope, retrieve, grade evidence, revise query, construct context, generate, verify claims, and terminate. A transition consumes typed state and produces a reason code. The state carries remaining budgets, evidence identities, query lineage, attempts, and policy. The model may propose a transition; trusted orchestration validates allowed edges and budgets. Avoid unconstrained recursive prompts that decide to “keep researching.”

Self-RAG introduces learned reflection tokens that allow a model to retrieve adaptively and critique retrieved passages and generation. CRAG introduces a retrieval evaluator and corrective actions, including alternate retrieval. These papers illustrate important design directions, not drop-in guarantees. A production implementation must define which decisions are learned, how they are calibrated, what evidence evaluators inspect, how failures surface, and which transitions are prohibited.

Evidence grading should separate relevance, authority, currency, and sufficiency. A passage can be relevant but obsolete, authoritative but incomplete, or individually useful while the set lacks a required facet. Use structured grades and thresholds validated on in-domain cases. A language model grader is itself stochastic and vulnerable to prompt injection in evidence. Limit its capabilities, label evidence as data, compare with deterministic metadata checks, and retain grading traces.

Query revision needs lineage and novelty control. Store parent query, transformation type, rationale, and result overlap. Reject a rewrite that drops mandatory entity, tenant, time, or jurisdiction constraints. Detect repeated normalized queries and high candidate overlap. A branch that adds no new relevant evidence consumes budget without progress. Set maximum depth and total branches.

Corrective retrieval can switch representation or source. If dense retrieval lacks an exact identifier, try lexical search. If the governed internal corpus has no evidence and policy permits, search an approved external corpus while clearly labelling authority. Do not broaden to the open web or a cross-tenant index merely because an evaluator says evidence is weak. Source expansion is an authorization decision.

Verification can trigger repair. Unknown citation handles require deterministic rejection. Unsupported claims can be removed, rewritten from cited evidence, or cause an additional retrieval branch focused on the missing claim. Cap repairs and prevent the generated claim itself from becoming trusted query context without sanitization. A contradicted high-consequence claim should escalate rather than loop until one passage appears agreeable.

Stopping conditions combine evidence and budgets. Stop answered when required facets have sufficient authorized support and claims verify. Stop abstained when routes are exhausted or evidence remains inadequate. Stop needs clarification when scope cannot be safely inferred. Stop conflict when authorities disagree. Stop budget when time, tokens, calls, or cost is exhausted. Every terminal state should be useful to the caller and observable to operators.

At toy scale, a state machine gets two retrieval attempts and cannot repeat a query. At application scale, a support assistant first searches current product docs, then an approved incident corpus if the question is diagnostic, and escalates if instructions conflict. At system scale, a research workflow runs parallel bounded branches, grades source coverage, joins evidence into a graph, verifies claims, and saves a replayable event journal. Larger scale does not justify less explicit control.

Evaluation needs trajectory metrics: task success, evidence coverage, unnecessary retrieval rate, branch utility, duplicate-query rate, correction success, verifier reversals, terminal-state accuracy, latency, cost, and policy violations. Compare with the fixed pipeline. An adaptive workflow that gains one answer point while tripling cost and increasing unsupported claims may not be an improvement. Slice by question complexity because simple queries should not pay for elaborate loops.

Fault-test the state machine. Inject retriever timeout, empty shard, corrupt evidence metadata, grader error, repeated rewrite, generator schema violation, and budget exhaustion. Assert terminal behavior, cleanup, no policy broadening, and complete lineage. The workflow becomes production-worthy when its failures are bounded and legible, not when a demo completes a long research trajectory.