AWS Builder Center

AWS Nitro Sandboxes: what it is, how to work with it, and how to set it up

AWS Nitro Sandboxes: what it is, how to work with it, and how to set it up

What it is and why it exists

AWS Nitro Sandboxes runs each workload in its own lightweight virtual machine with its own kernel. AWS  announced the feature  on November 9, 2026, for Amazon EKS, Amazon ECS and Amazon EC2.
The problem it solves is an old one. Containers on the same node share the host kernel, so a vulnerability in one container can expose its neighbors. The usual workaround was a dedicated machine for each sensitive workload, and most of that machine sat idle.
In its  EKS launch post , AWS gives four reasons the problem has become sharper:
  • AI agents write and run code that nobody reviews first.
  • SaaS platforms keep hundreds of customers on shared clusters.
  • Regulated industries need cryptographic proof, not configuration checklists.
  • CI/CD builds run with elevated privileges.
Nitro Sandboxes gives such a workload a hardware boundary without a dedicated machine. The isolation comes from the same Nitro hypervisor that separates EC2 instances. There is no separate charge: you pay for the EC2 instance that hosts the sandboxes.

How it works

A sandbox runs alongside the EC2 instance, not inside it. The instance is called the parent: it contributes some of its vCPUs and memory, and the Nitro hypervisor enforces the boundary between them ( EC2 User Guide ).
[embed: node/8c37e369-181d]
Read it left to right: kubelet hands the pod to the shim, which starts the sandbox through Firecracker. The hypervisor keeps the node kernel and the guest kernel apart.
On EKS, a pod starts in four steps ( EKS User Guide ):
  1. kubelet receives a pod with the  isolated-sandbox  or  confidential-sandbox  runtime class.
  1. The  containerd-shim-nitro-sandbox-v2  shim on the node creates a sandbox for the pod, which is a Firecracker microVM.
  1. An agent inside the sandbox evaluates the pod's policy and loads the images.
  1. Containers start inside the sandbox through its own containerd.
The sandbox is a first-level guest with direct hardware virtualization, with no emulation and no nested virtualization. Firecracker on the node only serves the virtual devices: disk, network and vsock. The sandbox's vCPUs are scheduled by Nitro, not by the node's kernel ( NOFire analysis ).

Two modes

You choose the mode at launch and cannot change it on a running sandbox.

Isolated
Confidential
Who is not trusted
The code in the pod
The code in the pod and the node itself
What is protected
The node and neighboring pods
Workload data from the node operator
 exec ,  stdin , process listing
Allowed
Denied
Pod policy
Accepts any valid container spec
Pins image digests, args, environment variables and mounts
Attestation
Not required
Verified through AWS KMS
EKS runtime class
 isolated-sandbox 
 confidential-sandbox 
The choice is the same at the EC2 level. Confidential mode, the default, removes the sandbox's memory from the parent instance. Direct mode leaves the memory accessible to the parent and fits cases where one trusted party operates both the sandbox and the instance.

When to choose Nitro Sandboxes

Nitro Sandboxes fits when independent workloads share your instance and the code and data must stay in your AWS account. Other cases have simpler or narrower tools.
Option
What it is
When to choose it
Nitro Sandboxes
A microVM on your instance, with AWS enforcing the boundary
AI agents running user code, workloads of different tenants, build jobs
Lambda MicroVMs
Serverless sandboxes from AWS
You do not want to manage instances or networking
Nitro Enclaves
An environment with no disk, external network or interactive access
A small job on highly sensitive data
Nested virtualization
Your own hypervisor inside the instance, Intel only
You need your own hypervisor or operating system
Kata Containers
Open-source microVMs for Kubernetes
The cluster is not on AWS; on EKS it needs nested virtualization
gVisor
A kernel in user space
Density matters more than full system call compatibility
E2B, Daytona, Modal, Vercel Sandbox
A sandbox as a service, called through an SDK
You need millisecond starts, pause with memory, GPUs, or have no platform team
The first four rows come from the  EC2 User Guide , the rest from the  Qovery comparison . The main trade-off: hosted services sell a finished sandbox, while Nitro Sandboxes gives you only the isolation boundary. You build the agent-facing API, warm pools and cleanup yourself.

How to work with it

A sandbox is enabled per pod, not per cluster. Set the runtime class and the pod runs in its own microVM ( Run a pod in isolated mode ).
apiVersion: v1
kind: Pod
metadata:
name: my-app
spec:
runtimeClassName: isolated-sandbox
containers:
- name: main
image: public.ecr.aws/amazonlinux/amazonlinux:2023
command: ["sleep", "3600"]
resources:
limits:
cpu: "1"
memory: 512Mi
To the application, the environment looks like a normal Linux container, so no code changes are needed. All containers in a pod share one sandbox. In isolated mode,  kubectl exec ,  logs  and  port-forward  work as usual.

Deployments and other controllers

Put  runtimeClassName  and every  sandbox.amazonaws.com/  annotation in the pod template:  spec.template.spec.runtimeClassName  and  spec.template.metadata.annotations . An annotation on the Deployment itself is accepted and then ignored.

Network limits

Every sandbox is throttled by default so that one pod cannot exhaust the node's shared resources ( Resource isolation and throttling ). All annotations use the  sandbox.amazonaws.com/  prefix.
Resource
Default
Annotation
Bandwidth, in and out
100 Mbps
 ingress-bandwidth ,  egress-bandwidth 
Packets, in and out
1,500 per second
 ingress-packets-per-second ,  egress-packets-per-second 
DNS queries
100 per second, 512 maximum
 dns-packets-per-second 
Outbound connections
4,096 per address family
 conntrack-limit 
Container logs
2 Mbps
 log-bandwidth 
Before a load test, raise both the bandwidth and the packet rate: they are enforced independently, and raising only one has no effect. Annotations are read once when the pod is created, so a new value needs a new pod. A value of  0  fails the pod; use  "disabled"  to turn a limit off.
metadata:
annotations:
sandbox.amazonaws.com/ingress-bandwidth: "1G"
sandbox.amazonaws.com/egress-bandwidth: "1G"
sandbox.amazonaws.com/ingress-packets-per-second: "15000"
sandbox.amazonaws.com/egress-packets-per-second: "15000"

Code with no network

For untrusted code that needs no network, set the annotation  sandbox.amazonaws.com/network-isolated: "true" . The sandbox boots with no network interface, DNS, IMDS or Pod Identity credentials ( Network isolation ).
This mode has three consequences:
  •  kubectl port-forward  does not work, and kubelet HTTP and TCP probes fail. Use exec probes.
  • Logs,  exec  and stats keep working over vsock.
  • The isolation is not absolute: the vsock channel is reachable by processes inside the guest.

Resources and density

A sandbox's CPU and memory are set at boot and cannot change while it runs. To change them, recreate the pod. Each pod also carries a fixed overhead: 256 MiB in isolated mode and 512 MiB in confidential mode.
A node holds up to five sandboxes per vCPU, and the sandboxes' total vCPUs can reach five times the node's vCPUs. In practice memory usually runs out first ( Considerations ).
On Intel and AMD instances with two threads per core, request CPU in multiples of two vCPUs. Compute there is bounded by physical cores, roughly half the advertised vCPUs. Graviton has one thread per core and is not affected.

Access and secrets

Isolation does not shrink IAM permissions: a pod with a broad Pod Identity role keeps that role inside the sandbox. Give each sandbox a role scoped to its task, or none at all ( Qovery comparison ). IAM credentials are not available through IMDS, which exposes only region and zone paths, so use EKS Pod Identity.
The  AWS coding agent example  is a good model. Its whole security policy comes down to three declared things:
  • An IAM role that allows only model invocation in Amazon Bedrock.
  • A short-lived token scoped to one GitHub repository.
  • A network policy that lets traffic out only to DNS, Bedrock and GitHub.

Storage

An EBS volume attaches to one pod only, with the  ReadWriteOncePod  access mode. For storage shared between pods, use Amazon EFS. You cannot expand an EBS volume while the pod is running.

Cold start and warm pools

AWS publishes no official cold-start numbers. In AWS samples, a cold start takes 5 to 6 seconds and a claim from a warm pool about one second, according to Qovery. For interactive sessions, AWS recommends the Kubernetes SIG agent-sandbox project: it keeps pre-booted VMs ready to claim in about two seconds.

Setup: EKS Auto Mode

EKS Auto Mode is the shortest path: sandbox support is built in and you build no AMI. The steps come from the  EKS launch post  and the  isolated mode guide .
  1. Create a cluster on Kubernetes 1.36 or later. With  eksctl , pass the version explicitly, because its default is older. eksctl create cluster --name sandbox-demo --region us-west-2 --version 1.37 --enable-auto-mode 
  1. Check that the runtime classes exist. The list should include  isolated-sandbox  and  confidential-sandbox . kubectl get runtimeclass 
  1. Add  runtimeClassName: isolated-sandbox  to the pod spec and apply the manifest. If no suitable node exists, Auto Mode provisions one. kubectl apply -f manifest.yaml kubectl get pod my-app 
  1. Confirm the pod runs in a VM. Inside a sandbox the product name ends in  .enclave , and the disks show up as  vda ,  vdb ,  vdc . kubectl exec my-app -- cat /sys/class/dmi/id/product_name kubectl exec my-app -- ls /sys/block 
If the pod stays in  Pending , check the Auto Mode constraints in  Considerations . Auto Mode does not provision a sandbox node in three cases:
  • The NodeClass uses FIPS AMIs.
  • The instance type's matching AMI is a GPU or Neuron variant.
  • The instance type has multiple NUMA nodes or has not yet been validated by AWS.
At launch, the  r5a ,  r5ad  and  m5ad  instance types are temporarily unavailable on Auto Mode.

Setup: standard EKS

Managed node groups, self-managed nodes and Karpenter need a custom AMI that carries the sandbox runtime. After that, pods use the same  runtimeClassName  as on Auto Mode ( Create and update a Nitro Sandbox AMI ).

What the AMI must contain

Component
Where it comes from
Host kernel with sandbox support
Amazon Linux 2023, kernel 6.18.51 or later
The  containerd-shim-nitro-sandbox-v2  shim
The signed runtime release from AWS
Guest image: an EIF and a disk image
Amazon S3, a matched pair per Kubernetes version and architecture
Firecracker and jailer
Release v1.18.0 or later from the open-source project
containerd
Version 2.0 or later, with  discard_unpacked_layers = false 
kubelet and nodeadm
Already in the EKS-optimized Amazon Linux 2023 AMI
You do not have to assemble this by hand. The  amazon-eks-ami  repository includes the  install-pod-isolation.sh  script, which builds such an AMI with Packer.

Steps

  1. Check the cluster version: you need Kubernetes 1.36 or later.
  1. Build the AMI on top of the EKS-optimized Amazon Linux 2023 AMI and add the components from the table.
  1. Reserve the sandbox memory pool with the  nitro_enclaves.mempool  kernel parameter, for example  95%-1G . Without the pool, a sandbox fails to start.
  1. Configure the runtime inside the AMI:
  • in  /etc/containerd/nitro-sandbox.toml , point  kernel_path  at the EIF file;
  • run the CRI proxy as a systemd service and point kubelet at  unix:///run/nitro-sandbox/cri-proxy.sock ;
  • register the  isolated-sandbox  and  confidential-sandbox  handlers in  /etc/containerd/config.toml .
  1. Launch nodes from this AMI on a supported instance type. In the launch template, set  EnclaveOptions.Enabled=true ; otherwise the node never advertises sandbox capacity.
  1. Label the nodes with  eks.amazonaws.com/nitro-sandbox=true , for example through  --node-labels .
  1. Apply the two  RuntimeClass  objects to the cluster once. apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: isolated-sandbox handler: isolated-sandbox overhead: podFixed: memory: 256Mi eks.amazonaws.com/nitro-sandbox: "1" scheduling: nodeSelector: eks.amazonaws.com/nitro-sandbox: "true" --- apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: confidential-sandbox handler: confidential-sandbox overhead: podFixed: memory: 512Mi eks.amazonaws.com/nitro-sandbox: "1" scheduling: nodeSelector: eks.amazonaws.com/nitro-sandbox: "true" 
  1. Run a pod with  runtimeClassName: isolated-sandbox  and verify it the same way as on Auto Mode.

Karpenter and updates

Karpenter needs version 1.14 or later: only from that version does  EC2NodeClass  have the  enclaveOptions  setting, according to  Qovery .
The AMI needs maintenance. When AWS publishes a new release, rebuild the AMI, update the launch template and roll the nodes one at a time. A runtime update needs no reboot; a kernel update does.

Setup: ECS and plain EC2

Amazon ECS

On ECS, sandboxes run only on ECS Managed Instances. ECS launches compatible EC2 instances, sets up a configured Bottlerocket image and runs each task in its own sandbox ( AWS Compute Blog ).
  1. Enable  nitroEnclavesEnabled  in the capacity provider settings.
  1. Add a  trustedExecutionConfiguration  block to the task definition.
  1. Set  isolationMode  in it to  ISOLATED_SANDBOX  or  CONFIDENTIAL_SANDBOX .
  1. Run the task and check the mode in the  trustedExecutionSummary  field of the task description.
These tasks use the  awsvpc  network mode, so each one gets its own network interface. Per-task security groups, VPC Flow Logs and load balancers work as before. The  ECS Developer Guide  has an overview of the feature.

Plain EC2

This path is for teams building their own platform or orchestrator. Your code then does what the service does on EKS and ECS: it launches sandboxes, connects their disks and network, accounts for resources and relays attestation.
  1. Launch a parent instance of a supported type with Nitro Enclaves enabled:  --enclave-options Enabled=true .
  1. Use a kernel with the sandbox driver, for example Amazon Linux 2023 with kernel 6.18.51 or later.
  1. Reserve the memory pool when the instance boots. Sandbox memory comes only from this pool.
  1. Add sandbox slots. The parent starts with 4 slots, more are added in steps of 8, and each step uses 128 MiB of the pool.
  1. Install Firecracker v1.18.0 or later. It is currently the only VMM that supports sandboxes.
  1. Prepare the guest image. A sandbox boots through UEFI, so the guest kernel must be a bzImage with an EFI stub.
  1. Describe the sandbox through the Firecracker  PUT /enclave  API and choose the mode: confidential or direct.
  1. Block the sandboxes from the parent's IMDS: reject traffic to  169.254.169.254 .
Step 8 matters for security. If the parent routes a sandbox's traffic through NAT, the sandbox can read the instance's IAM role from its IMDS. IMDSv2 and the hop limit do not prevent this ( EC2 User Guide ).
Keep three numbers in mind when sizing: at least 256 MiB of memory per sandbox, up to 32 virtual devices each, and at most 896 sandboxes per instance. The exact commands and configuration format are in the Run Nitro Sandboxes with Firecracker section of the same guide and in the  Firecracker documentation .

Confidential mode

Confidential mode is for workload data that must be protected from the node itself and its operators. It requires a Cedar policy and attestation conditions on your AWS KMS keys ( Run a pod in confidential mode ).

Three steps

  1. Generate a policy from the manifest. The  eks-nitro-sandbox-genpolicy  tool ships in the signed runtime release. It prints the policy in base64 and the predicted PCR17 value. eks-nitro-sandbox-genpolicy --profile confidential manifest.yaml 
  1. Add the policy to a pod annotation and set the  confidential-sandbox  runtime class. metadata: annotations: sandbox.amazonaws.com/policy: "<base64 policy>" spec: runtimeClassName: confidential-sandbox 
  1. Add attestation conditions to the KMS key policy and apply the manifest. KMS releases the key only when the measurements match. "Condition": { "StringEqualsIgnoreCase": { "kms:RecipientAttestation:PCR17": "<PCR17 from step 1>", "kms:RecipientAttestation:PCR8:ServiceName": "eks.amazonaws.com" } } 

What to remember

  • PCR17 measures the Cedar policy hash, PCR0 measures the system image, and PCR8 measures the certificate that signed the guest image.
  • PCR17 alone is not enough. Without the PCR8 condition, a policy with the same hash could be presented from an image that AWS did not sign.
  • Without attestation conditions in KMS, the measurements are recorded but never verified. In that case confidential mode does not protect secrets.
  • Generate the policy from the pod's final spec, after all mutating webhooks have run. Otherwise the spec no longer matches the policy and the pod is denied.
  • Any change to a pinned field, including a new image digest, needs a new policy and a new PCR17 value in KMS. Build this step into CI/CD.
To avoid regenerating the policy on every image rebuild, you can trust an AWS Signer signature instead of the digest. Generate the policy with the  --require-any-signer  flag. The pod's role then needs the  signer:GetRevocationStatus  and  signer:GetSigningProfile  permissions.
Disks have a subtlety of their own. The parent instance can read every block device it presents to a sandbox, including encrypted EBS volumes. To keep data confidential, encrypt it inside the sandbox with a key that KMS releases only after attestation ( EC2 User Guide ).

Limits and pitfalls

Most limits follow from one fact: the pod no longer shares a kernel with the node. Pods that do not use the features below run unchanged ( Considerations ).
Limit
What to do
GPU and device passthrough are not supported
Run those pods without a sandbox runtime class
CPU and memory cannot change while running
Recreate the pod with new requests
 hostNetwork: true  stops the pod from starting
Remove it from the spec
 hostUsers: false  is not supported
Remove it: the hardware boundary is already there
 hostPID  and  hostIPC  see only their own sandbox
Do not rely on node processes
An EBS volume cannot be shared between pods
Use  ReadWriteOncePod  or Amazon EFS
IMDS returns only region and zone
Use EKS Pod Identity for credentials
 port-forward  cannot reach a server on  127.0.0.1 
Listen on all interfaces or go through  kubectl exec 
 emptyDir  usage reports as zero
Set  ephemeral-storage  requests and limits
The CPU Manager  static  policy has no effect
Rely on CPU requests and limits
Four more limits come from the EC2 level and have no workaround ( EC2 User Guide ):
  • There is no snapshot and restore, pause and resume, or live migration. A sandbox lives exactly as long as its pod.
  • Stopping, rebooting or terminating the parent instance stops all of its sandboxes.
  • Not every instance type qualifies. Most families exclude the smallest size and  .metal , and flex instances are not supported at all.
  • Sandboxes cannot be enabled on an instance that has nested virtualization turned on.
There are no independent performance benchmarks yet. AWS states near-native EC2 performance but publishes no figures.

Sources

Official AWS material:
Third-party material:
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article