Plan queries and retrieve evidence

The user’s utterance is rarely a complete retrieval query. It may contain conversation references, ambiguous entities, multiple subquestions, time constraints, or an assumption that should not be accepted. Query planning converts the request and trusted session state into one or more bounded searches. The goal is evidence coverage with preserved lineage, not maximal query expansion.

First resolve scope. Identify actor and authorized corpora, task purpose, entity, product or jurisdiction, time, language, and desired answer form. Ask a clarifying question when a missing distinction materially changes evidence. A system should not guess whether “production” means an environment, a manufacturing process, or model serving if the corpus contains all three. Default scopes must be explicit and visible.

Represent the plan as data. A plan contains original request ID, subquery IDs, text, structured filters, route, candidate budget, dependencies, and reason. Derived queries point to their parent and transformation version. This lineage allows evaluators to determine which branch found or missed evidence. Do not let generated query text silently carry authority; filters come from trusted context or confirmed user input.

Useful transformations include acronym expansion, entity normalization, spelling alternatives, decomposition, and hypothetical-document embedding. HyDE prompts a language model to create a hypothetical relevant document and embeds that document to retrieve real documents, enabling zero-shot dense retrieval. The hypothetical text is a search probe, not evidence. It must never be cited or added to the authoritative corpus, and its claims should not enter the answer unless retrieved sources support them.

Decomposition helps multi-hop or comparative questions. “Compare retry policy before and after version 4” can branch into the two versions, then join results. Branching increases cost and the chance of spurious evidence. Set maximum branches, per-branch candidate depth, total retrieval deadline, and stopping criteria. Preserve unmodified full-query retrieval as a baseline because decomposition can remove context.

Conversation rewriting is another risk. Replace pronouns using validated conversational entities, but keep the original turn and rewrite. A malicious or mistaken earlier assistant message should not become a trusted filter. Allow users to inspect scope. Long sessions may contain contradictory requirements; retrieval plans should derive from current confirmed state rather than an unbounded transcript summary.

Candidate generation should combine signals according to the relevance contract. Lexical search protects exact names, clauses, and codes. Dense search connects paraphrases. Structured filters enforce policy, validity, and product version. Hybrid fusion and reranking operate over stable evidence identities. For RAG, candidate recall is often more important than final search-display precision because context construction will select a small set, but excessive noisy candidates raise reranking cost and poisoning exposure.

Multi-hop retrieval may alternate search and observation. A first document reveals an identifier needed for a second query. Treat this as a bounded state machine: retrieve, inspect structured evidence, derive a query, validate it, and stop at a depth or evidence criterion. A generator’s desire for “more research” is not a stopping policy. Cycles and repeated near-identical queries should be detected.

No-result and conflicting-result paths are normal. A no-result response can relax non-security filters, try a lexical or semantic fallback, ask a clarification, or abstain. Log which relaxation occurred. Never remove tenant, classification, or jurisdiction constraints merely to produce an answer. If current authoritative sources conflict, return the conflict with citations or escalate; ranking one slightly higher does not resolve governance.

Evaluate plans at branch and aggregate level. Measure relevant evidence found per branch, redundant branches, unsupported transformations, latency, candidate count, and whether clarification would have been better. Create adversarial fixtures with ambiguous names, misleading conversation state, prompt injection in prior text, and expensive decompositions. Compare plan policies against direct hybrid retrieval.

A disciplined planner reduces both misses and invented confidence. It transforms queries to find evidence, but only evidence can ground the answer. Every transformation remains a hypothesis with a budget, lineage, and evaluation record.