Secure Agent Sandboxes on EKS
Coding agents execute untrusted code. Standard containers share a host kernel. How to close that gap on EKS using gVisor, Kata Containers, and Firecracker.
The challenge with running agents in a pod
Coding agents start where developers always have, on a laptop. The agent runs locally and executes generated code right next to the developer's own tools and data. A bad run affects only that machine. Move that to a remote environment, though, and the security model changes. A dedicated VM limits the blast radius to that instance while a container on a shared host doesn't: a kernel exploit inside the container can reach every other workload on the node.
There are three common agentic patterns, and they don't all need the same level of isolation. The first is simple: an agentic loop where the LLM calls APIs and orchestrates tasks without executing arbitrary code. A standard container handles this fine. For the other two patterns, isolation matters more. In a remote sandbox, the agentic loop runs in a standard container but dispatches code execution to a separate, isolated environment. This allows us to maintain a strong security boundary between our loop and untrusted code. Most teams land here. The sandbox becomes a persistent, addressable environment a developer connects to from a laptop, a terminal, or a mobile device, picking up where they left off. The third pattern removes the boundary altogether with the LLM loop, orchestration, code generation, and execution all running inside the sandbox.
On paper, the last two patterns look slightly different, but in practice, they place the same demands on the execution environment. This sandbox must be isolated so a compromised session can't reach the host or other workloads, and should support snapshotting so the agent can restore to a known-good state or branch into parallel runs without rebuilding from scratch. Network egress and ingress filtering is a third requirement, restricting agents to approved endpoints. Credential management adds a related challenge with prompt injection turning any accessible credential into an exfiltration vector. Injecting credentials at the proxy level, outside the sandbox, limits that surface.
Workload isolation is where the standard container model falls short. CVE-2024-21626 , a 2024 runc container escape via a file descriptor leak, showed the consequences: a process inside a container compromised every pod on the affected node. Namespaces and cgroups provide process and resource isolation, but they do not provide a separate kernel. For untrusted or user-generated code, that boundary is not enough. You need a stronger isolation primitive: a dedicated kernel per session, whether through EC2 instances or isolated runtimes within a Kubernetes cluster.
Isolation without rebuilding from scratch
The options for running isolated agent workloads sit on a spectrum. On the managed end, services like AWS Bedrock AgentCore, E2B Sandboxes, and Vercel Sandbox handle isolation, the control plane, and operational complexity. You provide the agent code and they handle the rest. E2B also open-sources its infrastructure, so you can run their platform and primitives in your own environment outside of their managed solution. On the self-built end, dedicated EC2 instances provide isolation at the VM level but require you to manage scheduling, scaling, and session lifecycle. Firecracker , an open-source virtual machine monitor (VMM) built by AWS in Rust to power services like Lambda , takes a different approach: thousands of lightweight microVMs on a single host. Deploying Firecracker directly on EC2 achieves the same kernel isolation at higher density, but adds a virtualization layer you must build and maintain: VM provisioning, scheduling, network routing, and snapshotting.
Kubernetes addresses the orchestration problem, but it doesn't solve kernel isolation by itself. That requires a runtime that provides it explicitly. Two runtimes provide it within Kubernetes, and they work in different ways.
gVisor places a Sentry process between each container and the host kernel. Sentry intercepts every syscall and handles it in userspace, so the host kernel never sees raw application syscalls. This is process-level isolation, not hardware isolation. The overhead runs 10–30% for I/O-intensive workloads and under 5% end-to-end for agents that spend most of their time waiting on LLM responses. Modal's Sandboxes and Anthropic's Claude Code remote execution environments both successfully use gVisor for agent workload isolation in production.
Kata Containers goes further. Each pod boots inside a dedicated Firecracker microVM with a real Linux guest kernel, and the boundary is hardware-enforced via KVM and Intel VT-x or AMD-V. A potential kernel exploit compromises the guest kernel, not the host. The combined stack of EKS, Kata, and Firecracker adds minimal memory overhead per microVM, with cold start around 5 seconds including pod scheduling. Breaking out requires escaping Firecracker itself, a VMM written in roughly 50,000 lines of Rust versus QEMU's two million lines of C.
Until February 16, 2026, Kata Containers with Firecracker required bare-metal EC2 instances, since Firecracker depends on KVM and KVM requires direct hardware access. EC2 nested virtualization on C8i, M8i, and R8i instances removed that constraint. Both runtimes can now run on standard virtual EC2 instances and integrate into an existing EKS cluster without changes to the application or agent being hosted.
Kubernetes exposes this choice through
RuntimeClass , a resource that maps a workload to a specific container runtime. You build the sandbox infrastructure once and select the runtime at session creation time based on the threat model. The workload definition stays identical regardless of which isolation boundary is in use.Building Sandcastles: an EKS-native multi-tenant Agent sandbox platform
Built on open-source primitives from the Kubernetes Apps SIG community , this design covers similar connection patterns as managed solutions like E2B or AgentCore using a self-managed approach.
I'll walk through the design starting with the agent-sandbox open-source project. It provides a Sandbox CRD and controller for isolated, stateful workloads. Templates embed the
RuntimeClass reference, so choosing gVisor or Kata is a configuration change, not a code change. Warm pools handle faster sandbox provisioning.I built the Sandbox Manager on top because agents that need a remote secure execution environment should not need Kubernetes API access. The Manager exposes a lifecycle API (create, delete, status) and the sandbox router proxies requests to the right pod by sandbox ID, where an exec agent handles command execution and file I/O. Client SDKs and CLIs talk to these APIs without knowing anything about the underlying cluster. For background agents, I simply host the complete agent inside the sandbox. The LLM loop, code generation, and execution all run within the same pod, with no router or exec agent in its path. The sandbox terminates when the agent stops.
I've validated the remote sandbox architecture with two clients. First, a coding agent built on Claude's Agent SDK and Claude Sonnet 4.6 via Amazon Bedrock provisioned a sandbox using its tools, built a sample application, ran its tests, and exited. Separately, OpenCode running locally on a developer's laptop connected to an OpenCode server inside the sandbox, giving the developer a fully remote coding environment with a clear isolation boundary. Persistent Volumes and the associated Persistent Volume Claim configuration on the sandbox template makes sessions persistent, so developers can disconnect and resume without keeping pods running.
Running both runtimes in the same cluster revealed a few takeaways. gVisor added negligible launch latency and required nothing beyond a standard EKS setup: no special configuration beyond installing the runtime shim. Kata Containers with Firecracker carried a heavier cost. Cold start was around 5 seconds per sandbox, and EKS Managed Node Groups do not work yet: they override the CPU Options stanza needed for nested virtualization, forcing the use of self-managed node groups and the maintenance overhead that comes with them.
Why isolation alone is not enough
Better isolation helps with reducing the blast radius, but it does not help with recovery. Without snapshotting, a failed run means rebuilding from scratch.
The primary use case is fast session creation: Build a complex environment once (dependencies, datasets, caches), snapshot it, and restore each new agent session from that snapshot in milliseconds rather than rebuilding from a base image. But snapshotting unlocks more than that. Snapshot before a risky operation and roll back in under a second if the outcome is wrong. Fork into N concurrent sandboxes to run different approaches in parallel. Capture the exact sandbox state at the point of failure so anyone can reproduce it without reconstructing the environment.
The managed platforms that ship snapshotting all build on the same primitive: Firecracker's native snapshot API, which pauses a running microVM, serializes its memory state to disk, and restores where it left off.
The Kata compatibility shim does not expose Firecracker's snapshot API, and gVisor has no snapshotting mechanism of its own. If you need snapshotting support, you are building your own Firecracker control plane, at which point Kubernetes becomes redundant as the orchestration layer. That is the complexity managed platforms abstract away, and why some skip Kubernetes entirely.
Beyond isolation: egress and credentials
Isolation defines the blast radius, but two related challenges remain: controlling what leaves the sandbox, and managing credentials inside it. Agents have open-ended egress to LLM APIs, package registries, version control, and arbitrary shell commands. Kubernetes NetworkPolicy operates on IP addresses, not domain names, so domain-based filtering requires additional tooling. Credentials are part of the same problem. Any secret visible inside the sandbox is accessible to a prompt-injected agent, and proxy-level credential injection narrows that exposure but doesn't eliminate it. Both topics warrant their own post.
Firecracker beyond Kubernetes
Managed sandbox platforms that run agent workloads at scale skip Kubernetes entirely. E2B and Vercel Sandbox provision Firecracker microVMs directly, managing their own schedulers, warm VM pools, and snapshot pipelines. The result is sandbox creation in under a second, versus the ~5 seconds Kata with Firecracker on EKS requires. The difference is not Firecracker itself: it is the Kata compatibility layer, which translates Kubernetes pod semantics into Firecracker microVM operations and adds 500ms–1s of startup latency before any application code runs.
The enabling conditions to close that gap on EKS now exist. EC2's support for nested virtualization on select instances makes Firecracker viable on standard EC2 instances without bare metal. Kubernetes already provides the scheduling, warm pool, and lifecycle primitives needed for sandbox orchestration. What it lacks is a native mechanism to use Firecracker's API directly. That capability exists today only in managed platforms like E2B and Vercel Sandbox, which build their own Firecracker control planes outside Kubernetes.
If you are building on EKS today, start with gVisor. It runs on any EC2 instance, deploys in an afternoon, and handles isolation for most agent workloads. When the threat model demands a hardware boundary, add a Kata Containers with Firecracker node group and accept the cold start overhead. Direct Firecracker provisioning from Kubernetes closes the remaining speed gap. That is the next piece to build.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article