Chapter 4 of 10
Graph evidence acquisition and claim ledgers
Knowledge retrieval
About 4 minutes · includes examples, an exercise, and references
Chapter at a glance
- •Use an append-only typed evidence ledger rather than relying on chat history.
- •Preserve ordered graph paths and claim-level support for derived conclusions.
- •Distinguish direct evidence, tool observations, user assertions, and model hypotheses.
A knowledge agent should accumulate evidence, not merely conversation text. The evidence ledger is a task-local, append-only record of source claims, graph paths, tool observations, generated hypotheses, contradictions, and decisions. It lets the controller determine what is known, what is derived, what remains uncertain, and which artifacts justify the next action.
Each evidence item has a stable identifier, kind, content or reference, provenance, acquisition action, time, authorization scope, freshness, and status. Kinds include direct source claim, graph-derived relation, tool observation, model hypothesis, user assertion, policy decision, and verification result. These kinds have different authority. A model hypothesis can guide a search but cannot satisfy a production precondition until verified.
Graph search returns structured paths, not just prose. A path includes ordered entity and relation identifiers, supporting claim IDs for every edge, direction, time, and graph release. The agent may use a path to infer impact if relation semantics allow composition. It should record the derived conclusion separately, with the inference rule and support set. A shortest path is not necessarily the strongest explanation; rank directness, relation type, source quality, temporal validity, and evidence diversity.
Evidence acquisition follows information needs. The planner states a question such as “Is Service S owned by Team T under the current release?” The retrieval action specifies authorized graph scope, predicate allow list, time, result limit, and required provenance. The observation reports answers and completeness metadata. Broad unrestricted subgraph dumps make prompt injection, leakage, and context overflow more likely.
Contradictions remain explicit. Two claims can disagree because of source conflict, time, scope, identity error, or extraction error. Do not overwrite the earlier ledger entry. Link a contradiction relation, retrieve disambiguating evidence, and apply a field-specific precedence policy only when one exists. The plan can branch into review or abstention.
Deduplicate by semantic identity while retaining distinct sources. If three documents repeat the same upstream announcement, the ledger can group the normalized claim but must not count three independent confirmations. Track derivation chains or near-duplicate source clusters. Confidence should consider source independence, not raw repetition.
Context assembly selects a view of the ledger for each model call. Provide the smallest relevant evidence set plus unresolved questions and constraints. Keep stable IDs so the model cites ledger items in proposals. Reject proposals that cite missing, unauthorized, stale, or hypothesis-only items for a precondition requiring direct evidence.
The ledger also supports progress. A transition makes progress when it resolves an information need, adds materially new authorized evidence, verifies a claim, or reduces a contradiction. Repeated paraphrases do not count. Evidence budgets limit items, graph hops, source domains, and tokens. When the budget ends, produce a result qualified by unresolved needs.
Privacy and retention apply to the ledger. It may combine sources in a way that increases sensitivity. Encrypt it, scope access to the task, minimize copied text, and expire it under policy. Audit logs can retain hashes and decisions without indefinitely retaining full sensitive context where rules permit.
Evaluate evidence precision and recall, path validity, provenance completeness, contradiction detection, independence, freshness, authorization, context utilization, and whether action preconditions cite acceptable evidence kinds. Replay should rebuild the same ledger from recorded retrieval actions and pinned releases, or explicitly identify external data that has changed.
Key points
- Use an append-only typed evidence ledger rather than relying on chat history.
- Preserve ordered graph paths and claim-level support for derived conclusions.
- Distinguish direct evidence, tool observations, user assertions, and model hypotheses.
- Count independent sources and progress, not repeated paraphrases.
A graph evidence ledger item
Read the expected behavior in the surrounding walkthrough, then copy and run this reference implementation.
{
"evidenceId": "ev-109",
"kind": "graph_path",
"question": "Which workflow depends on package P-9?",
"path": {
"entities": ["package/P-9", "service/S-2", "workflow/W-3"],
"relations": ["usedBy", "supports"],
"claimIds": ["claim-21", "claim-27"]
},
"graphRelease": "kg-2025-04-18.2",
"scope": "engineering-public",
"validAt": "2025-04-18T12:00:00Z",
"status": "verified"
}Worked examples
Toy
Hypothesis cannot satisfy a gate
The model hypothesizes that Team A owns a service.
Store it as a hypothesis and issue an ownership query. Only a verified registry observation can satisfy the owner precondition for a change request.
- Evidence kind
- Information need
- Verification transition
Application
Conflicting dependency paths
A current manifest and an old architecture document disagree.
Keep both paths with valid time and source roles. A precedence rule favors the live manifest for current deployment state while the historical document remains cited for design intent.
- Temporal scope
- Source authority by field
- No destructive overwrite
Exercise
Implement an evidence-ledger loop
Answer a two-hop graph question with one contradiction and one missing precondition.
- Define evidence kinds and authority rules.
- Store path claims and provenance.
- Generate and resolve information needs.
- Enforce evidence and no-progress budgets.
Success criteria
- Every derived conclusion cites all supporting edges.
- Hypotheses cannot satisfy verified gates.
- Contradictions remain visible after policy selection.
- Replay identifies the graph release and acquisition action.
Reflect: Which item looked like evidence only because it appeared in a confident model response?
References and further reading
- PROV-O: The PROV OntologyThe W3C Recommendation for representing entities, activities, agents, derivations, and attribution.
- SPARQL 1.1 Query LanguageThe W3C Recommendation for graph patterns, filters, aggregation, property paths, subqueries, and RDF datasets.
- Microsoft GraphRAG Local SearchOfficial documentation for entity-seeded retrieval over graph structures, source text, claims, and community reports.