Reading tools and contents
Harness Engineering & Sandboxes

Chapter 4 of 10

Containers, application kernels, and microVM isolation

Isolation

About 4 minutes · includes examples, an exercise, and references

Chapter at a glance

  • Containers share the host kernel; microVMs and application kernels alter that exposure.
  • Filesystem, network, credentials, artifacts, and control plane are part of the isolation boundary.
  • Resource containment requires killable process domains and independent limits.

Isolation starts with a threat model, not a product name. Ask who controls the code, which secrets and tenants share the host, what kernel attack surface is acceptable, which side channels matter, and what compatibility and startup latency the workload requires. A namespace container, an application-kernel sandbox, and a hardware-virtualized microVM expose different boundaries. Selecting among them is a risk and systems decision; stacking them can provide defense in depth.

An OCI container packages a process with declared namespaces, mounts, capabilities, resource settings, and lifecycle hooks. Namespaces isolate views of processes, mounts, users, and networks; cgroups account for and constrain resources; capabilities split portions of root authority; seccomp can reduce system calls. These controls are substantial, but ordinary containers share the host kernel. A kernel vulnerability reachable through the allowed system-call surface can cross the intended boundary. Running as a non-root user, dropping capabilities, using a read-only root filesystem, denying host mounts, and applying seccomp are baseline hardening, not optional polish.

Application-kernel systems such as gVisor interpose a userspace kernel between the workload and the host kernel. The gVisor Sentry implements a Linux-like interface and exposes a restricted host surface. This can reduce shared-kernel attack exposure, but it introduces compatibility and performance tradeoffs, particularly for system-call-heavy workloads or uncommon kernel features. The official gVisor security model is explicit about what it does not protect, including some higher-layer configuration failures and hardware side channels. A harness must preserve those caveats rather than marketing the sandbox as absolute.

MicroVMs such as Firecracker use hardware virtualization and a minimal virtual machine monitor. The guest has its own kernel, so arbitrary guest processes do not directly issue system calls to the host kernel. Firecracker deliberately minimizes emulated devices and startup overhead for high-density workloads. Stronger boundary does not mean zero risk: VMM bugs, device interfaces, host configuration, shared hardware, image supply chain, and control-plane authorization remain. MicroVMs also impose image management, boot, networking, snapshot, and observability complexity.

Isolation includes data paths. Use a copy-on-write workspace rather than mounting the host repository read-write. Deliver inputs through a staged immutable channel. Export outputs through a broker that validates paths, sizes, types, and ownership. Credentials should be task-scoped, short-lived, injected only after authorization, and unavailable to child processes unless required. Network policy should default deny. Allow-list destinations through an egress proxy that resolves DNS safely, prevents private-address confusion, caps bytes, and records response identity.

Process limits must exist inside and outside the guest. A timeout in the orchestrator stops elapsed time but may not stop descendants if cancellation is incomplete. The runtime should place all workload processes in a killable resource domain, send graceful termination if appropriate, enforce a hard deadline, and verify no descendants survive. CPU, memory, process count, file descriptors, filesystem bytes, output bytes, and network bytes need independent limits. cgroup v2 provides hierarchical controls but cannot replace application-level budgets such as tool calls or external spend.

Snapshots and warm pools improve latency but create state-reuse hazards. A warmed environment must not retain tenant data, credentials, random generator state, trace context, or modified policy. Prefer snapshots made before tenant material is introduced. If reuse occurs, run a measurable sanitation protocol and assign a new identity. Periodically destroy the pool and verify that forensic scans find no cross-run residue.

The control plane is part of the boundary. If an attacker can ask the runner to mount arbitrary host paths, grant devices, use privileged mode, or attach production credentials, a strong microVM is irrelevant. Validate the runtime configuration against an allow-list and record it as part of the fixture. Separate the service that decides policy from the worker that launches workloads. Workers should receive signed or authenticated run specifications and hold only the authority needed to instantiate them.

Measure isolation choices with representative workloads and adversarial tests. Compatibility, cold-start latency, steady-state overhead, density, syscall profile, and cleanup time are empirical properties. Security review should enumerate exposed interfaces rather than assigning a single “secure” score. The correct boundary for trusted internal data transformations may differ from the boundary for anonymous generated code in a multi-tenant service. The harness should make that choice explicit per task class.

Key points

  • Containers share the host kernel; microVMs and application kernels alter that exposure.
  • Filesystem, network, credentials, artifacts, and control plane are part of the isolation boundary.
  • Resource containment requires killable process domains and independent limits.
  • Warm reuse must prove tenant-state sanitation.

Worked examples

Toy

Untrusted expression

A classroom expression evaluator accepts no imports or filesystem access.

Use a restricted parser and process quota; do not launch a general shell merely because the expression is small.

  • Language-level restriction precedes OS isolation.
  • Timeout and output caps are independently tested.
  • No ambient environment variables reach the process.

Application

Generated data-science script

A known employee runs generated Python against a sanitized dataset.

Use a hardened rootless container or application-kernel sandbox with a read-only base, per-run data mount, no default network, and bounded artifact export.

  • Threat actor and data sensitivity justify the boundary.
  • Package installation is separated from execution.
  • Exported files cross a validation broker.

System

Anonymous multi-tenant code runner

Internet users submit arbitrary binaries to shared infrastructure.

Use microVM isolation, per-run guest kernel, minimal devices, strict host control plane, egress broker, layered quotas, and post-run destruction.

  • Host and guest patch ownership is explicit.
  • Control-plane requests cannot enable privileged configuration.
  • Cross-tenant residue and side channels are assessed.

Exercise

Choose an isolation boundary

Compare two execution workloads with different trust and compatibility requirements.

  1. Write attacker, asset, and escape-impact statements for each workload.
  2. Compare hardened container, gVisor, and microVM boundaries using exposed interfaces.
  3. Define an adversarial validation and performance benchmark.

Success criteria

  • The choice follows the threat model rather than branding.
  • Data and control-plane paths are included.
  • Known non-goals and operational costs are recorded.

Reflect: Which host interface remains reachable in your proposed design, and why is that acceptable?

References and further reading