Chapter 7 of 10

Hybrid retrieval and adaptive traversal

Retrieval engineering

About 4 minutes · includes examples, an exercise, and references

Chapter at a glance

•Choose channels from query features and evaluate the router independently.
•Use rank fusion rather than assuming heterogeneous scores are comparable.
•Treat adaptive retrieval as a bounded evidence-gathering loop.

No single retrieval channel dominates every question. Lexical search is strong for exact identifiers, rare terms, quoted phrases, and error codes. Dense retrieval captures semantic similarity. Graph search follows typed relations and distributed evidence. Community reports support broad synthesis. Hybrid retrieval makes these channels compete and cooperate under an explicit plan instead of assuming one universal index.

Begin with query features that can be computed safely: detected identifiers, named entities, relation verbs, requested time, global cues such as “across all,” expected answer type, and ambiguity. A deterministic router can handle obvious cases; a model router can propose a plan through a validated schema. Always keep a fallback and log the selected route. The router itself needs evaluation because an excellent local retriever cannot answer a global question routed to it.

Parallel hybrid search runs several channels and fuses their ranked lists. Reciprocal rank fusion combines rank positions without pretending lexical, vector, and graph scores share a scale. Its constant controls how strongly top ranks dominate. Weighted fusion can reflect query class, but weights must be learned or selected on held-out data. Retrieve identifiers, channel rank, release, and evidence—not only snippets—so duplicates can be recognized across channels.

Sequential hybrid search uses one channel to seed another. Lexical retrieval can discover an exact project code, then graph traversal expands dependencies. Dense retrieval can find source units, whose resolved entities become graph seeds. A community report can orient a global query, followed by local searches for claims that need verification. Sequential plans risk error propagation, so retain multiple seeds and add stop conditions.

DRIFT search combines a community-level primer with iterative local exploration. The general pattern is adaptive retrieval: create a provisional answer or information-needs list, choose the next evidence action, update state, and stop when coverage, confidence, cost, or iteration limits are reached. This resembles an agent loop and requires the same controls. The model may propose queries, but only allow-listed search operations execute.

An adaptive state should include the original question, resolved constraints, evidence already seen, unresolved subquestions, contradictions, per-channel budgets, and provenance. Novelty scoring prevents repeatedly retrieving paraphrases. Coverage checks compare collected evidence with an answer schema or expected subtopics. Stop because a measurable condition is met, not because the generated text sounds complete.

Reranking should use features aligned with support: directness, source authority for the field, recency, relation-path validity, evidence diversity, and authorization. A cross-encoder or model judge can improve relevance, but its score is another generated artifact. Evaluate it separately and retain the pre-rerank candidate set for replay.

Hybrid systems amplify cache complexity. Cache keys need normalized query, route, index and graph releases, embedding and reranker versions, time constraints, and authorization scope. A cached unrestricted candidate list cannot safely be filtered for a restricted caller after ranking because hidden artifacts may have shaped order and summaries.

Measure channel contribution through ablations and oracle analysis. Ablation removes one channel and observes retrieval and answer changes. Oracle analysis asks whether any channel contained the needed evidence even when fusion missed it. Track router accuracy, candidate recall, fused precision, evidence diversity, iterations, tokens, latency, and answer support by query slice. A channel that rarely wins may still be essential for rare high-consequence queries.

Complexity must earn its place. Prefer the simplest plan meeting the quality objective. A two-channel fused retriever with stable citations may outperform an adaptive loop at a fraction of cost and operational risk. Promote an adaptive strategy only when it shows reproducible gains and bounded failure behavior.

Key points

Choose channels from query features and evaluate the router independently.
Use rank fusion rather than assuming heterogeneous scores are comparable.
Treat adaptive retrieval as a bounded evidence-gathering loop.
Use ablations and oracle analysis to prove each channel’s contribution.

Reciprocal rank fusion with channel provenance

Read the expected behavior in the surrounding walkthrough, then copy and run this reference implementation.

Reciprocal rank fusion with channel provenancepython

from collections import defaultdict

def reciprocal_rank_fusion(result_lists, k=60):
    scores = defaultdict(float)
    channels = defaultdict(list)
    for channel, results in result_lists.items():
        for rank, item_id in enumerate(results, start=1):
            scores[item_id] += 1.0 / (k + rank)
            channels[item_id].append((channel, rank))
    ordered = sorted(scores, key=scores.get, reverse=True)
    return [{"id": item, "score": scores[item], "channels": channels[item]} for item in ordered]

Worked examples

Toy

Error-code route

The query contains an exact rare error code and asks what systems are affected.

Lexical search finds the defining ticket; its service entity seeds a two-hop graph expansion. Dense results add semantically related recovery notes, and fusion preserves channel provenance.

Router feature
Sequential seed
Cross-channel duplicate

Application

Adaptive policy comparison

A user asks how two policies differ across obligations, exceptions, and effective dates.

Start with dense policy sections, resolve clause entities, traverse supersession and exception edges, and stop when each comparison field has cited evidence or is explicitly unknown.

Coverage schema
Iteration budget
Unknown field

Exercise

Evaluate a hybrid retrieval plan

Build lexical, dense, and graph candidate lists for a mixed set of local and global questions.

Define router features and fallbacks.
Fuse ranked lists with provenance.
Add one bounded adaptive plan.
Run per-channel ablation and oracle analysis.

Success criteria

Score fusion is reproducible.
Adaptive steps have explicit budgets and stop rules.
Cache keys include authorization and index versions.
Each retained channel has measured incremental value.

Reflect: Which sophisticated channel was compensating for a fixable extraction or chunking defect?