Fault injection and adversarial resilience experiments

A happy-path harness test proves that expected components can cooperate once. It does not prove the system is safe when dependencies delay, workers die, outputs truncate, storage fills, policies change, or an observation contains adversarial instructions. Fault injection turns failure assumptions into repeatable experiments. The objective is evidence about invariants and recovery, not spectacle or random breakage.

Start with a steady-state hypothesis stated in user-visible or safety terms. Examples: no duplicate external write occurs; every accepted run eventually reaches a terminal and cleaned state; tenant artifacts never cross boundaries; cancellation prevents new effects within a bound; and evaluator failures never become success. Define observable measures and a time window. Then inject one controlled variable and attempt to falsify the hypothesis.

Build a fault catalog by boundary. Planner faults include malformed JSON, unavailable model, rate limit, long latency, repeated proposals, and misleading self-reported success. Tool faults include schema drift, partial output, nonzero exit, huge output, transport loss, and ambiguous commit. Worker faults include process kill, host restart, memory pressure, disk-full, clock shift, and cleanup failure. Control-plane faults include duplicate delivery, stale lease, unavailable policy service, revoked approval, and telemetry exporter failure. Environmental inputs can include prompt injection, poisoned files, symlink traps, decompression bombs, and malicious archive paths.

Inject at trusted seams rather than scattering random sleeps. Adapter interfaces can accept a fault schedule keyed by run and invocation. Network dependencies can route through Toxiproxy to add latency, reset connections, limit bandwidth, or cut links deterministically. Workers can expose a test-only crash point after effect and before journal commit—the dangerous window that ordinary unit tests miss. Filesystems can use bounded volumes or explicit failing wrappers.

Fault experiments need blast-radius controls. Run destructive cases in isolated test tenants with synthetic credentials and hard quotas. Start with unit and integration environments, then carefully use canaries where production-only interactions matter. Define abort conditions, maximum duration, affected percentage, and a responsible operator. Principles of Chaos Engineering emphasize measurable steady state and minimizing blast radius; “chaos” is not permission to run uncontrolled failures.

Pair every injected fault with an oracle. If the network drops after a write, inspect the downstream idempotency record. If the worker dies, verify lease expiration, adoption, reconciliation, and cleanup. If logs cannot export, verify the durable journal remains intact and metrics surface exporter loss. If an observation contains instructions, verify the adapter labels it as untrusted data and policy prevents new authority. Passing means the stated invariant holds, not merely that the service returns.

Use fault matrices rather than combinatorial explosion. Select representative task classes, trust levels, and lifecycle phases. Prioritize by impact, likelihood, and uncertainty. Run pairwise combinations for coupled risks such as timeout plus duplicate delivery, disk pressure plus cleanup, and prompt injection plus a high-privilege tool proposal. Preserve seeds and schedules so a failure becomes a regression test.

Adversarial evaluation overlaps but is not identical. Fault injection tests infrastructure and control invariants under known perturbations. Red teaming searches for unexpected strategies and semantic failures. A robust program uses both. Findings should become minimized fixtures where possible, with the original trace retained for context and a small deterministic test added to CI.

Measure recovery quality, not only availability. Time to detection, time to containment, cleanup completion, evidence completeness, duplicate-effect count, budget overrun, and operator burden reveal whether the harness fails safely. A run that eventually succeeds after silently exceeding cost or leaving credentials active is not resilient.

Finally, test the fault system. Ensure injection cannot activate in unauthorized production scope, fault labels cannot be forged by ordinary callers, and experiment telemetry is distinguishable from organic incidents. The mechanism that can kill workers or cut networks is itself a consequential capability with approval, audit, and budgets.