Hybrid candidate generation and rank fusion

Hybrid retrieval combines systems whose errors are not perfectly correlated. A lexical retriever catches exact terms, codes, and names. A dense retriever catches paraphrases and conceptual relations. Learned sparse or late-interaction retrievers contribute other evidence. The objective is not to average everything; it is to improve recall and rank on defined query slices while keeping latency, cost, and behavior understandable.

Start by running retrievers independently and unioning candidates by stable passage identity. Retain rank and raw score from each source. A document absent from one list is missing evidence, not necessarily a zero on the other model’s scale. Component scores often have incompatible distributions. BM25 scores vary with query terms and collection statistics; cosine values depend on the embedding model; learned scores may be logits. Naive weighted addition without calibration can let one component dominate unpredictably.

Reciprocal Rank Fusion avoids raw score calibration. For each candidate d, sum 1/(k + rank_r(d)) across retrievers r. The constant k reduces the impact of very high ranks and controls how much deeper list positions contribute. A candidate ranked well by several systems rises; a candidate rescued by one system still enters the union. RRF is simple and robust, but it discards score margins. Tune k and candidate depths on judged queries, and retain component traces.

Score fusion can be appropriate when calibrated. Options include per-query min-max normalization, z-scores, learned logistic calibration, or a learning-to-rank model using component scores and query features. Each has risks. Min-max is sensitive to outliers and list truncation. Z-scores assume a useful distribution. Learned fusion can overfit domains or query mixes. Train and evaluate with temporal splits where possible, because query and corpus distributions evolve.

Query routing can reduce cost. An exact quoted identifier may rely heavily on lexical search; a natural-language conceptual question may allocate more dense candidates; an empty or unsupported query may return guidance instead of searching. Routing should be a measurable policy, not an opaque shortcut. Log route features and compare against an always-run baseline to quantify saved cost and missed recall.

Filters must be applied consistently. If both indexes support authorization and version filters, apply them within candidate generation. If one cannot, do not expose its unauthorized candidates to later stages or logs. Divergent analyzers and document generations create confusing unions, so every result should carry corpus and representation versions. Deduplicate by canonical search-unit identity, not normalized text alone; identical text in two authorized sources may have different provenance.

Candidate depth is a budget. Increasing lexical and dense k can raise recall but increases ANN work, union size, network transfer, and reranking cost. Allocate depth by measured marginal gain. Plot candidate recall as each source depth grows, then inspect overlap. If two retrievers return nearly identical sets, the second may add little. If a critical slice relies entirely on one retriever, protect it with a release gate.

Hybrid debugging is a stage ledger. For a query and judged relevant unit, record whether it existed in the published corpus, passed filters, appeared in each candidate list, survived union and deduplication, received each fusion contribution, entered reranking, and appeared in the final list. This turns “hybrid search feels worse” into a localized defect.

At toy scale, two short ranked lists make RRF transparent. At application scale, support search unions BM25 identifiers with semantic symptoms, reranks the top forty, and returns five distinct articles. At system scale, a federated service queries regional lexical and vector shards, enforces tenant policy locally, fuses only compatible evidence, and declares partial coverage when a shard fails. The same fusion equation is the easy part; identity, policy, versioning, and observability make it reliable.

Release a hybrid system only after an ablation study. Compare lexical only, dense only, union without reranking, fusion variants, and the complete pipeline. Report overall metrics, slices, latency, cost, and confidence intervals. Complexity is justified when a component rescues important cases without unacceptable regression elsewhere.