LLM decision guide
Long Context vs Retrieval
Long context supplies more text directly; retrieval selects a smaller evidence set for each request.
Choose Long context when
- The corpus is small and request-specific
- Cross-document synthesis needs broad visibility
Choose Retrieval when
- The corpus is large or permissioned
- Freshness, citations, or cost matter
Decision table
| Dimension | Long context | Retrieval |
|---|---|---|
| Selection | Mostly before request | Query-time ranking |
| Cost | Grows with supplied tokens | Adds index and retrieval cost |
| Failure | Distraction and position effects | Missed or misranked evidence |
Example
A realistic choice
A single contract may fit long context; a changing enterprise policy library usually needs retrieval.
Hands-on
Test the decision
Estimate token cost and evidence recall for both designs on the same ten-question set.