Chapter 1 of 8
Agency is a controlled feedback process
Chapter 1
About 5 minutes · includes examples, an exercise, and references
Chapter at a glance
- •Treat the model as a policy component and the runtime as the authority-bearing controller.
- •Represent goal, state, action, observation, verifier, budgets, and terminal reason explicitly.
- •Choose the minimum useful autonomy and require independent evidence before accepting completion.
An agentic AI system is not merely a language model asked to produce a longer answer. It is software that repeatedly chooses an action from a bounded action space, observes the consequence, updates explicit state, and stops for a verifiable reason. The model may propose actions, plans, or interpretations, but application code owns authority, execution, persistence, and termination. This distinction is the foundation for every reliable design decision that follows.
Begin with five nouns. A goal describes a desired outcome. State records facts the runtime currently accepts, including the request, prior observations, budgets, approvals, and unresolved questions. An action is a typed proposal selected from a finite set such as search, read, calculate, ask, write, or finish. An environment contains systems outside the model: databases, files, web services, people, queues, clocks, and other agents. An observation is a structured report of what happened after an action. A sixth noun, the verifier, decides whether evidence satisfies the goal and whether a claimed terminal state is legitimate.
These nouns separate three systems that are often blurred together. The model is a probabilistic policy over possible outputs. The runtime is a deterministic controller that validates model output, calls tools, updates state, applies budgets, and records events. The product is the human and organizational context that defines permissions, acceptable failure, escalation, and value. A model can be capable while the runtime is unsafe. A runtime can be correct while the product goal is poorly specified. Treating the bundle as “the agent” hides the boundary at which each defect belongs.
Agency exists on a spectrum. A one-shot classifier has no meaningful action loop. A model that may choose between two read-only retrieval tools has limited agency. A coding assistant that can edit files, run tests, install packages, and publish a pull request has broader agency because actions change durable state. Greater agency is not automatically better. It expands the reachable state space, the consequences of mistakes, and the evidence needed before release. The appropriate design question is therefore not “How autonomous can this be?” but “What is the least autonomy that completes the task within the risk envelope?”
The observe–decide–act–verify cycle is a useful mental picture. Observe converts raw environment data into bounded, typed facts. Decide asks the policy for one next action, not an unbounded fantasy plan. Act executes through a capability that enforces authorization, validation, timeout, and resource limits. Verify compares the new state with explicit success and safety predicates. If verification fails, the loop may gather more evidence, repair a recoverable problem, ask a human, or stop. Verification is not the model writing “done”; it is an independent check such as a passing test suite, a reconciled record count, a cited source, or a human approval receipt.
ReAct demonstrated the value of interleaving reasoning and actions rather than treating planning and interaction as separate phases. The engineering lesson is broader than any prompting format: decisions should be revisable when observations invalidate assumptions. Do not store hidden prose as the only state. Extract decisions, tool calls, results, and terminal reasons into records that can be inspected and replayed. Reasoning text can help a model, but operational truth belongs in typed state.
The first production invariant is authority separation: model output is data until trusted code validates and authorizes it. The second is boundedness: every run has limits on steps, elapsed time, model tokens, tool calls, external writes, and cost. The third is evidence-based completion: the runtime accepts success only when an independently computed predicate holds. The fourth is legibility: an operator can reconstruct which inputs, policies, model versions, actions, and observations produced the outcome.
Consider a support agent asked to refund an order. The goal is not “make the user happy”; it is a contract such as “resolve the eligible refund request without exceeding policy or duplicating a payment.” State includes authenticated customer identity, order identifier, policy version, current order status, refund history, approval requirements, and budgets. Read actions may inspect the order and policy. A write action may create a refund only through an idempotent service. Verification reads the payment ledger and confirms one refund with the intended amount. An ambiguous order, policy conflict, or high-value amount produces an escalation terminal state, not improvisation.
A good agent specification can be read without knowing which model will be used. It names the goal, state schema, action catalogue, observation schema, invariants, budgets, verification rules, and terminal reasons. Model choice then affects quality, latency, and cost inside that contract. If changing the model changes what the system is allowed to do, authority was placed in the wrong layer.
Key points
- Treat the model as a policy component and the runtime as the authority-bearing controller.
- Represent goal, state, action, observation, verifier, budgets, and terminal reason explicitly.
- Choose the minimum useful autonomy and require independent evidence before accepting completion.
- Make every consequential decision reconstructable from typed events rather than hidden reasoning alone.
A minimal observe–decide–act–verify loop
Read the expected behavior in the surrounding walkthrough, then copy and run this reference implementation.
from dataclasses import dataclass
from typing import Literal
Observation = Literal["higher", "lower", "equal"]
@dataclass(frozen=True)
class State:
low: int = 1
high: int = 20
steps: int = 0
def decide(state: State) -> int:
return (state.low + state.high) // 2
def observe(secret: int, guess: int) -> Observation:
return "equal" if guess == secret else "higher" if secret > guess else "lower"
def reduce_state(state: State, guess: int, observation: Observation) -> State:
if observation == "higher":
return State(guess + 1, state.high, state.steps + 1)
if observation == "lower":
return State(state.low, guess - 1, state.steps + 1)
return State(state.low, state.high, state.steps + 1)
def run(secret: int, max_steps: int = 5) -> tuple[str, State]:
state = State()
while state.steps < max_steps and state.low <= state.high:
action = decide(state)
result = observe(secret, action)
if result == "equal":
return "verified_success", reduce_state(state, action, result)
state = reduce_state(state, action, result)
return "budget_exhausted", state
assert run(17)[0] == "verified_success"
assert run(21)[0] == "budget_exhausted"
print(run(17))Worked examples
Toy
Find a number with a bounded oracle
An agent must identify a secret integer from 1 through 20 using only higher, lower, and equal observations.
State stores the remaining interval. The policy selects its midpoint. The environment returns a typed comparison. Verification accepts success only on equal, while a step budget prevents repeated guesses. The toy case exposes state reduction, observations, and termination without model complexity.
- The interval shrinks after every valid observation.
- No action can leave the state unchanged twice.
- Success is determined by the oracle, not by the policy.
Application
Policy-bounded refund assistant
A support workflow reads an order, checks a versioned refund policy, requests approval when required, and creates at most one refund.
The model interprets the request and proposes tools. The runtime authenticates identity, validates arguments, checks permissions, and records an idempotency key. A ledger read verifies the final effect. Policy ambiguity becomes an escalation rather than another guess.
- Authenticated principal and policy version are explicit.
- Read and write capabilities are separate.
- The refund is verified against the ledger.
System
Repository repair agent
A coding agent works in an isolated checkout, searches code, edits files, runs tests, and proposes a patch without direct deployment access.
The environment is a disposable sandbox. State includes repository revision, patch, test evidence, tool budget, and trace identifiers. Completion requires targeted tests plus regression checks; publication is a separately approved capability.
- The base revision and environment are reproducible.
- Test evidence is attached to the exact patch.
- Publishing is outside the default authority set.
Exercise
Write an agent contract before a prompt
Choose a real task that might benefit from agency and specify its control contract without mentioning a model vendor.
- Define a measurable goal and at least three terminal reasons: success, bounded failure, and escalation.
- List state fields, allowed actions, observation fields, and which component owns each transition.
- Define two safety invariants, hard budgets, and an independent completion check.
- Remove one capability and explain whether the smaller authority set still completes the task.
Success criteria
- The contract distinguishes proposals from authorized effects.
- Every action has a bounded observation and a named owner.
- Completion can be checked without trusting a natural-language claim.
- The design explains why its degree of autonomy is necessary.
Reflect: Which failure becomes impossible when authority moves from the model into the runtime?