Reading tools and contents
4 min
Harness Engineering & Sandboxes

Chapter 3 of 10

Typed tool contracts, authorization, and bounded observations

Interfaces

About 4 minutes · includes examples, an exercise, and references

Chapter at a glance

  • •Validate syntax, semantics, authorization, and budget as separate gates.
  • •Bind permissions to principal, tenant, capability, effect, and target.
  • •Return typed bounded observations with artifact references and truncation metadata.

Tools are security and reliability interfaces, not convenient function names. A tool contract must be precise enough that untrusted proposed arguments can be rejected before effects occur and narrow enough that authorization can be evaluated without interpreting prose. The model may decide which operation would help; trusted application code decides whether the operation is valid, authorized, affordable, and safe to execute.

Define inputs with a closed schema. Prefer discriminated unions over bags of optional fields. Put limits on string length, array size, numeric range, path depth, result count, and recursive structures. Reject unknown fields when they can hide intent. Separate identifiers from display names. A file adapter should accept a workspace-relative normalized path, not an arbitrary URI. A network adapter should accept a destination capability issued by policy, not a model-supplied raw host if egress is constrained.

Schema validation establishes shape, not meaning. Business invariants need a second validation layer. A transfer amount can be a valid positive number and still exceed the caller’s authority. A command can be a valid string and still invoke a shell. A patch can be valid text and still touch forbidden files. Run semantic checks after parsing and before execution. Return structured errors that distinguish invalid input, denied authorization, exhausted budget, dependency failure, and internal adapter failure. Do not leak secrets or policy internals in the error.

Authorization should bind four things: authenticated principal, tenant, capability, and target resource. It should also include effect class and relevant context such as approval state. Read and write variants should be separate capabilities even if one library implements both. High-consequence operations should support prepare/commit. The prepare call validates and returns a human-readable preview plus a short-lived commitment token. Commit verifies that the principal, arguments, policy version, and target have not changed.

Idempotency belongs in the contract for effects that may be retried. The caller supplies or receives an idempotency key scoped to the tenant and operation. The adapter stores the first committed outcome and returns it for duplicate delivery. Idempotency is not achieved by asking the model not to repeat itself. For non-idempotent legacy APIs, add a reconciliation operation that checks external state before retrying and records ambiguity instead of guessing.

Outputs need schemas and limits too. Tool output is untrusted environmental data that may contain malformed structures, huge logs, binary content, or instructions aimed at the model. Return a compact observation with status, typed payload, truncation metadata, artifact references, and a redacted diagnostic. Store large artifacts outside the context window under content identity. Include enough provenance to retrieve them through a separate authorized path. Treat natural-language text in observations as data, not higher-priority instructions.

Version contracts explicitly. Additive optional output fields can often be backward compatible; changed semantics, narrower authorization, or different default effects require a new version or negotiated capability. Record the contract version on every invocation. MCP, OpenAPI, and JSON Schema offer established vocabulary for describing messages and interfaces, but a harness still needs local policies for identity, effects, budgets, and observation retention.

Adapter implementation should isolate parsing, policy, effect, and serialization. The effect function should receive a validated domain object rather than raw JSON. It should not read ambient credentials or environment settings that were absent from the capability. Where possible, make adapters small enough for property tests: invalid paths never escape the workspace, output is always capped, denied calls produce no effect, and duplicate keys do not duplicate commits.

The final contract is the audit event. Record invocation identity, contract version, validated argument digest, policy decision, effect start and end, result class, artifact identities, redaction status, and resource charge. Sensitive arguments may require selective hashing or encrypted evidence. Never put raw secrets in model-visible observations or ordinary traces. A debugging system that leaks credentials is not a trustworthy harness.

Good tool design changes model behavior indirectly. Narrow tools reduce the space of invalid choices, structured errors make repair possible, and preview/commit separates planning from consequences. These are software guarantees. They remain valuable when the model is replaced, prompts change, or an adversarial document enters context.

Key points

  • Validate syntax, semantics, authorization, and budget as separate gates.
  • Bind permissions to principal, tenant, capability, effect, and target.
  • Return typed bounded observations with artifact references and truncation metadata.
  • Make retries safe with idempotency or explicit reconciliation.

Validate a closed tool request

Read the expected behavior in the surrounding walkthrough, then copy and run this reference implementation.

Validate a closed tool requestpython
from dataclasses import dataclass
from pathlib import PurePosixPath

@dataclass(frozen=True)
class ReadRequest:
    path: str
    max_bytes: int

def validate_read(raw: dict) -> ReadRequest:
    if set(raw) != {"path", "max_bytes"}:
        raise ValueError("unknown or missing fields")
    if not isinstance(raw["path"], str) or not raw["path"]:
        raise ValueError("path must be a non-empty string")
    if type(raw["max_bytes"]) is not int:
        raise ValueError("max_bytes must be an integer")
    path = PurePosixPath(raw["path"])
    if path.is_absolute() or ".." in path.parts:
        raise ValueError("path must remain workspace-relative")
    limit = raw["max_bytes"]
    if not 1 <= limit <= 64_000:
        raise ValueError("max_bytes outside policy")
    return ReadRequest(path.as_posix(), limit)

print(validate_read({"path": "src/main.py", "max_bytes": 4096}))

Exercise

Threat-model a tool contract

Choose one consequential tool and redesign it as a narrow capability.

  1. Write closed input and output schemas with quantitative limits.
  2. Define semantic, authorization, idempotency, and redaction rules.
  3. List property tests that prove denied or malformed calls have no effect.

Success criteria

  • No raw shell, unrestricted URL, or ambient path field remains.
  • Retry behavior is deterministic or explicitly ambiguous.
  • Observations are bounded and separate data from control.

Reflect: What authority does your current tool inherit that is absent from its public schema?

References and further reading