The knowledge agent as a bounded transition system

A knowledge-driven agent is a system that observes a task and environment, retrieves governed knowledge, proposes actions, executes authorized tools, and updates explicit state until it reaches a terminal condition. The language model may interpret goals and propose steps, but the agent is the whole controlled transition system. This framing replaces the vague idea of “an autonomous model” with components that can be typed, tested, paused, replayed, and denied.

Let state contain the original goal, normalized constraints, plan, evidence ledger, tool observations, pending approvals, budgets, and lifecycle status. An action is a typed request with preconditions and expected effects. The environment executes or rejects it and returns an observation. A transition function constructs the next state from prior state, selected action, and observation. Termination occurs on success, safe failure, explicit denial, timeout, cancellation, or budget exhaustion.

ReAct demonstrated the utility of interleaving reasoning and action, while Toolformer studied learned tool invocation. A production design should borrow the pattern without treating free-form reasoning text as executable control. The model emits a structured proposal; deterministic policy validates schema, permissions, preconditions, idempotency, and budget; a tool adapter executes; an evidence recorder commits the observation; and a controller selects the next allowed transition.

Distinguish knowledge from memory. Governed domain knowledge includes ontology terms, source claims, policies, and graph releases. Working memory is task-local state. Episodic memory records previous trajectories. Semantic memory may store reusable facts or summaries derived from experience. These stores have different authority, retention, freshness, and injection risks. A previous agent answer is not promoted to domain truth merely because it was useful once.

Represent observations as data with provenance. A search result identifies query, index release, source unit, and scope. A graph result identifies claim and path evidence. A tool response identifies request, adapter, external version or transaction, and time. Untrusted text inside an observation cannot redefine the agent’s goal, permissions, or tool policy. The controller consumes typed fields and passes content to the model inside a marked evidence boundary.

Budgets create bounded autonomy. Set limits on model calls, tool calls, elapsed time, tokens, retrieval depth, write count, monetary cost, and repeated failures. Budgets belong to controller state and are decremented outside the model. An agent cannot grant itself more budget by proposing an action. Use per-tool and per-risk limits: twenty read operations may be acceptable while one production write requires approval.

Terminal conditions need evidence. “Done” is not whatever the model says. A task contract defines success assertions and how to verify them: a ticket exists with specified fields, a graph query returns a validated release, tests pass, or a reviewer approves a proposal. When verification is impossible, the correct result is a qualified report or escalation, not fabricated completion.

Loops require progress checks. Compare successive states for new evidence, resolved subgoals, changed environment, or reduced uncertainty. Repeating equivalent searches or retries without state change triggers a stop or alternate plan. Tool errors are typed as retryable, permanent, authorization, invalid-request, or ambiguous-outcome; each category has a bounded policy.

The simplest architecture is often enough: one controller, one state object, a tool registry, a policy layer, and an append-only event log. Multi-agent arrangements add coordination, authorization, and consistency costs. Split roles only when independent context, parallel work, or separation of duties produces measurable value.

The core engineering question is not whether the model can imagine a solution. It is whether every meaningful transition is authorized, evidence-bearing, bounded, and recoverable.