AWS Nitro Sandboxes: what it is, how to work with it, and how to set it up
AWS Nitro Sandboxes: what it is, how to work with it, and how to set it up
What it is and why it exists
AWS Nitro Sandboxes runs each workload in its own lightweight virtual machine with its own kernel. AWS announced the feature on November 9, 2026, for Amazon EKS, Amazon ECS and Amazon EC2.
The problem it solves is an old one. Containers on the same node share the host kernel, so a vulnerability in one container can expose its neighbors. The usual workaround was a dedicated machine for each sensitive workload, and most of that machine sat idle.
- AI agents write and run code that nobody reviews first.
- SaaS platforms keep hundreds of customers on shared clusters.
- Regulated industries need cryptographic proof, not configuration checklists.
- CI/CD builds run with elevated privileges.
Nitro Sandboxes gives such a workload a hardware boundary without a dedicated machine. The isolation comes from the same Nitro hypervisor that separates EC2 instances. There is no separate charge: you pay for the EC2 instance that hosts the sandboxes.
How it works
A sandbox runs alongside the EC2 instance, not inside it. The instance is called the parent: it contributes some of its vCPUs and memory, and the Nitro hypervisor enforces the boundary between them ( EC2 User Guide ).
[embed: node/8c37e369-181d]
Read it left to right: kubelet hands the pod to the shim, which starts the sandbox through Firecracker. The hypervisor keeps the node kernel and the guest kernel apart.
- kubelet receives a pod with the
isolated-sandboxorconfidential-sandboxruntime class.
- The
containerd-shim-nitro-sandbox-v2shim on the node creates a sandbox for the pod, which is a Firecracker microVM.
- An agent inside the sandbox evaluates the pod's policy and loads the images.
- Containers start inside the sandbox through its own containerd.
The sandbox is a first-level guest with direct hardware virtualization, with no emulation and no nested virtualization. Firecracker on the node only serves the virtual devices: disk, network and vsock. The sandbox's vCPUs are scheduled by Nitro, not by the node's kernel ( NOFire analysis ).
Two modes
You choose the mode at launch and cannot change it on a running sandbox.
| Isolated | Confidential |
|---|---|---|
Who is not trusted | The code in the pod | The code in the pod and the node itself |
What is protected | The node and neighboring pods | Workload data from the node operator |
exec , stdin , process listing | Allowed | Denied |
Pod policy | Accepts any valid container spec | Pins image digests, args, environment variables and mounts |
Attestation | Not required | Verified through AWS KMS |
EKS runtime class | isolated-sandbox | confidential-sandbox |
The choice is the same at the EC2 level. Confidential mode, the default, removes the sandbox's memory from the parent instance. Direct mode leaves the memory accessible to the parent and fits cases where one trusted party operates both the sandbox and the instance.
When to choose Nitro Sandboxes
Nitro Sandboxes fits when independent workloads share your instance and the code and data must stay in your AWS account. Other cases have simpler or narrower tools.
Option | What it is | When to choose it |
|---|---|---|
Nitro Sandboxes | A microVM on your instance, with AWS enforcing the boundary | AI agents running user code, workloads of different tenants, build jobs |
Lambda MicroVMs | Serverless sandboxes from AWS | You do not want to manage instances or networking |
Nitro Enclaves | An environment with no disk, external network or interactive access | A small job on highly sensitive data |
Nested virtualization | Your own hypervisor inside the instance, Intel only | You need your own hypervisor or operating system |
Kata Containers | Open-source microVMs for Kubernetes | The cluster is not on AWS; on EKS it needs nested virtualization |
gVisor | A kernel in user space | Density matters more than full system call compatibility |
E2B, Daytona, Modal, Vercel Sandbox | A sandbox as a service, called through an SDK | You need millisecond starts, pause with memory, GPUs, or have no platform team |
The first four rows come from the EC2 User Guide , the rest from the Qovery comparison . The main trade-off: hosted services sell a finished sandbox, while Nitro Sandboxes gives you only the isolation boundary. You build the agent-facing API, warm pools and cleanup yourself.
How to work with it
A sandbox is enabled per pod, not per cluster. Set the runtime class and the pod runs in its own microVM ( Run a pod in isolated mode ).
apiVersion: v1kind: Podmetadata: name: my-appspec: runtimeClassName: isolated-sandbox containers: - name: main image: public.ecr.aws/amazonlinux/amazonlinux:2023 command: ["sleep", "3600"] resources: limits: cpu: "1" memory: 512MiTo the application, the environment looks like a normal Linux container, so no code changes are needed. All containers in a pod share one sandbox. In isolated mode,
kubectl exec , logs and port-forward work as usual.Deployments and other controllers
Put
runtimeClassName and every sandbox.amazonaws.com/ annotation in the pod template: spec.template.spec.runtimeClassName and spec.template.metadata.annotations . An annotation on the Deployment itself is accepted and then ignored.Network limits
Every sandbox is throttled by default so that one pod cannot exhaust the node's shared resources ( Resource isolation and throttling ). All annotations use the
sandbox.amazonaws.com/ prefix.Resource | Default | Annotation |
|---|---|---|
Bandwidth, in and out | 100 Mbps | ingress-bandwidth , egress-bandwidth |
Packets, in and out | 1,500 per second | ingress-packets-per-second , egress-packets-per-second |
DNS queries | 100 per second, 512 maximum | dns-packets-per-second |
Outbound connections | 4,096 per address family | conntrack-limit |
Container logs | 2 Mbps | log-bandwidth |
Before a load test, raise both the bandwidth and the packet rate: they are enforced independently, and raising only one has no effect. Annotations are read once when the pod is created, so a new value needs a new pod. A value of
0 fails the pod; use "disabled" to turn a limit off.metadata: annotations: sandbox.amazonaws.com/ingress-bandwidth: "1G" sandbox.amazonaws.com/egress-bandwidth: "1G" sandbox.amazonaws.com/ingress-packets-per-second: "15000" sandbox.amazonaws.com/egress-packets-per-second: "15000"Code with no network
For untrusted code that needs no network, set the annotation
sandbox.amazonaws.com/network-isolated: "true" . The sandbox boots with no network interface, DNS, IMDS or Pod Identity credentials ( Network isolation ).This mode has three consequences:
-
kubectl port-forwarddoes not work, and kubelet HTTP and TCP probes fail. Use exec probes.
- Logs,
execand stats keep working over vsock.
- The isolation is not absolute: the vsock channel is reachable by processes inside the guest.
Resources and density
A sandbox's CPU and memory are set at boot and cannot change while it runs. To change them, recreate the pod. Each pod also carries a fixed overhead: 256 MiB in isolated mode and 512 MiB in confidential mode.
A node holds up to five sandboxes per vCPU, and the sandboxes' total vCPUs can reach five times the node's vCPUs. In practice memory usually runs out first ( Considerations ).
On Intel and AMD instances with two threads per core, request CPU in multiples of two vCPUs. Compute there is bounded by physical cores, roughly half the advertised vCPUs. Graviton has one thread per core and is not affected.
Access and secrets
Isolation does not shrink IAM permissions: a pod with a broad Pod Identity role keeps that role inside the sandbox. Give each sandbox a role scoped to its task, or none at all ( Qovery comparison ). IAM credentials are not available through IMDS, which exposes only region and zone paths, so use EKS Pod Identity.
The AWS coding agent example is a good model. Its whole security policy comes down to three declared things:
- An IAM role that allows only model invocation in Amazon Bedrock.
- A short-lived token scoped to one GitHub repository.
- A network policy that lets traffic out only to DNS, Bedrock and GitHub.
Storage
An EBS volume attaches to one pod only, with the
ReadWriteOncePod access mode. For storage shared between pods, use Amazon EFS. You cannot expand an EBS volume while the pod is running.Cold start and warm pools
AWS publishes no official cold-start numbers. In AWS samples, a cold start takes 5 to 6 seconds and a claim from a warm pool about one second, according to Qovery. For interactive sessions, AWS recommends the Kubernetes SIG agent-sandbox project: it keeps pre-booted VMs ready to claim in about two seconds.
Setup: EKS Auto Mode
EKS Auto Mode is the shortest path: sandbox support is built in and you build no AMI. The steps come from the EKS launch post and the isolated mode guide .
- Create a cluster on Kubernetes 1.36 or later. With
eksctl, pass the version explicitly, because its default is older.eksctl create cluster --name sandbox-demo --region us-west-2 --version 1.37 --enable-auto-mode
- Check that the runtime classes exist. The list should include
isolated-sandboxandconfidential-sandbox.kubectl get runtimeclass
- Add
runtimeClassName: isolated-sandboxto the pod spec and apply the manifest. If no suitable node exists, Auto Mode provisions one.kubectl apply -f manifest.yaml kubectl get pod my-app
- Confirm the pod runs in a VM. Inside a sandbox the product name ends in
.enclave, and the disks show up asvda,vdb,vdc.kubectl exec my-app -- cat /sys/class/dmi/id/product_name kubectl exec my-app -- ls /sys/block
If the pod stays in
Pending , check the Auto Mode constraints in Considerations . Auto Mode does not provision a sandbox node in three cases:- The NodeClass uses FIPS AMIs.
- The instance type's matching AMI is a GPU or Neuron variant.
- The instance type has multiple NUMA nodes or has not yet been validated by AWS.
At launch, the
r5a , r5ad and m5ad instance types are temporarily unavailable on Auto Mode.Setup: standard EKS
Managed node groups, self-managed nodes and Karpenter need a custom AMI that carries the sandbox runtime. After that, pods use the same
runtimeClassName as on Auto Mode ( Create and update a Nitro Sandbox AMI ).What the AMI must contain
Component | Where it comes from |
|---|---|
Host kernel with sandbox support | Amazon Linux 2023, kernel 6.18.51 or later |
The containerd-shim-nitro-sandbox-v2 shim | The signed runtime release from AWS |
Guest image: an EIF and a disk image | Amazon S3, a matched pair per Kubernetes version and architecture |
Firecracker and jailer | Release v1.18.0 or later from the open-source project |
containerd | Version 2.0 or later, with discard_unpacked_layers = false |
kubelet and nodeadm | Already in the EKS-optimized Amazon Linux 2023 AMI |
You do not have to assemble this by hand. The amazon-eks-ami repository includes the
install-pod-isolation.sh script, which builds such an AMI with Packer.Steps
- Check the cluster version: you need Kubernetes 1.36 or later.
- Build the AMI on top of the EKS-optimized Amazon Linux 2023 AMI and add the components from the table.
- Reserve the sandbox memory pool with the
nitro_enclaves.mempoolkernel parameter, for example95%-1G. Without the pool, a sandbox fails to start.
- Configure the runtime inside the AMI:
- in
/etc/containerd/nitro-sandbox.toml, pointkernel_pathat the EIF file;
- run the CRI proxy as a systemd service and point kubelet at
unix:///run/nitro-sandbox/cri-proxy.sock;
- register the
isolated-sandboxandconfidential-sandboxhandlers in/etc/containerd/config.toml.
- Launch nodes from this AMI on a supported instance type. In the launch template, set
EnclaveOptions.Enabled=true; otherwise the node never advertises sandbox capacity.
- Label the nodes with
eks.amazonaws.com/nitro-sandbox=true, for example through--node-labels.
- Apply the two
RuntimeClassobjects to the cluster once.apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: isolated-sandbox handler: isolated-sandbox overhead: podFixed: memory: 256Mi eks.amazonaws.com/nitro-sandbox: "1" scheduling: nodeSelector: eks.amazonaws.com/nitro-sandbox: "true" --- apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: confidential-sandbox handler: confidential-sandbox overhead: podFixed: memory: 512Mi eks.amazonaws.com/nitro-sandbox: "1" scheduling: nodeSelector: eks.amazonaws.com/nitro-sandbox: "true"
- Run a pod with
runtimeClassName: isolated-sandboxand verify it the same way as on Auto Mode.
Karpenter and updates
Karpenter needs version 1.14 or later: only from that version does
EC2NodeClass have the enclaveOptions setting, according to Qovery .The AMI needs maintenance. When AWS publishes a new release, rebuild the AMI, update the launch template and roll the nodes one at a time. A runtime update needs no reboot; a kernel update does.
Setup: ECS and plain EC2
Amazon ECS
On ECS, sandboxes run only on ECS Managed Instances. ECS launches compatible EC2 instances, sets up a configured Bottlerocket image and runs each task in its own sandbox ( AWS Compute Blog ).
- Enable
nitroEnclavesEnabledin the capacity provider settings.
- Add a
trustedExecutionConfigurationblock to the task definition.
- Set
isolationModein it toISOLATED_SANDBOXorCONFIDENTIAL_SANDBOX.
- Run the task and check the mode in the
trustedExecutionSummaryfield of the task description.
These tasks use the
awsvpc network mode, so each one gets its own network interface. Per-task security groups, VPC Flow Logs and load balancers work as before. The ECS Developer Guide has an overview of the feature.Plain EC2
This path is for teams building their own platform or orchestrator. Your code then does what the service does on EKS and ECS: it launches sandboxes, connects their disks and network, accounts for resources and relays attestation.
- Launch a parent instance of a supported type with Nitro Enclaves enabled:
--enclave-options Enabled=true.
- Use a kernel with the sandbox driver, for example Amazon Linux 2023 with kernel 6.18.51 or later.
- Reserve the memory pool when the instance boots. Sandbox memory comes only from this pool.
- Add sandbox slots. The parent starts with 4 slots, more are added in steps of 8, and each step uses 128 MiB of the pool.
- Install Firecracker v1.18.0 or later. It is currently the only VMM that supports sandboxes.
- Prepare the guest image. A sandbox boots through UEFI, so the guest kernel must be a bzImage with an EFI stub.
- Describe the sandbox through the Firecracker
PUT /enclaveAPI and choose the mode: confidential or direct.
- Block the sandboxes from the parent's IMDS: reject traffic to
169.254.169.254.
Step 8 matters for security. If the parent routes a sandbox's traffic through NAT, the sandbox can read the instance's IAM role from its IMDS. IMDSv2 and the hop limit do not prevent this ( EC2 User Guide ).
Keep three numbers in mind when sizing: at least 256 MiB of memory per sandbox, up to 32 virtual devices each, and at most 896 sandboxes per instance. The exact commands and configuration format are in the Run Nitro Sandboxes with Firecracker section of the same guide and in the Firecracker documentation .
Confidential mode
Confidential mode is for workload data that must be protected from the node itself and its operators. It requires a Cedar policy and attestation conditions on your AWS KMS keys ( Run a pod in confidential mode ).
Three steps
- Generate a policy from the manifest. The
eks-nitro-sandbox-genpolicytool ships in the signed runtime release. It prints the policy in base64 and the predicted PCR17 value.eks-nitro-sandbox-genpolicy --profile confidential manifest.yaml
- Add the policy to a pod annotation and set the
confidential-sandboxruntime class.metadata: annotations: sandbox.amazonaws.com/policy: "<base64 policy>" spec: runtimeClassName: confidential-sandbox
- Add attestation conditions to the KMS key policy and apply the manifest. KMS releases the key only when the measurements match.
"Condition": { "StringEqualsIgnoreCase": { "kms:RecipientAttestation:PCR17": "<PCR17 from step 1>", "kms:RecipientAttestation:PCR8:ServiceName": "eks.amazonaws.com" } }
What to remember
- PCR17 measures the Cedar policy hash, PCR0 measures the system image, and PCR8 measures the certificate that signed the guest image.
- PCR17 alone is not enough. Without the PCR8 condition, a policy with the same hash could be presented from an image that AWS did not sign.
- Without attestation conditions in KMS, the measurements are recorded but never verified. In that case confidential mode does not protect secrets.
- Generate the policy from the pod's final spec, after all mutating webhooks have run. Otherwise the spec no longer matches the policy and the pod is denied.
- Any change to a pinned field, including a new image digest, needs a new policy and a new PCR17 value in KMS. Build this step into CI/CD.
To avoid regenerating the policy on every image rebuild, you can trust an AWS Signer signature instead of the digest. Generate the policy with the
--require-any-signer flag. The pod's role then needs the signer:GetRevocationStatus and signer:GetSigningProfile permissions.Disks have a subtlety of their own. The parent instance can read every block device it presents to a sandbox, including encrypted EBS volumes. To keep data confidential, encrypt it inside the sandbox with a key that KMS releases only after attestation ( EC2 User Guide ).
Limits and pitfalls
Most limits follow from one fact: the pod no longer shares a kernel with the node. Pods that do not use the features below run unchanged ( Considerations ).
Limit | What to do |
|---|---|
GPU and device passthrough are not supported | Run those pods without a sandbox runtime class |
CPU and memory cannot change while running | Recreate the pod with new requests |
hostNetwork: true stops the pod from starting | Remove it from the spec |
hostUsers: false is not supported | Remove it: the hardware boundary is already there |
hostPID and hostIPC see only their own sandbox | Do not rely on node processes |
An EBS volume cannot be shared between pods | Use ReadWriteOncePod or Amazon EFS |
IMDS returns only region and zone | Use EKS Pod Identity for credentials |
port-forward cannot reach a server on 127.0.0.1 | Listen on all interfaces or go through kubectl exec |
emptyDir usage reports as zero | Set ephemeral-storage requests and limits |
The CPU Manager static policy has no effect | Rely on CPU requests and limits |
- There is no snapshot and restore, pause and resume, or live migration. A sandbox lives exactly as long as its pod.
- Stopping, rebooting or terminating the parent instance stops all of its sandboxes.
- Not every instance type qualifies. Most families exclude the smallest size and
.metal, and flex instances are not supported at all.
- Sandboxes cannot be enabled on an instance that has nested virtualization turned on.
There are no independent performance benchmarks yet. AWS states near-native EC2 performance but publishes no figures.
Sources
Official AWS material:
- Introducing AWS Nitro Sandboxes on Amazon EKS, ECS and EC2 , AWS What's New, November 9, 2026
- Isolate workloads with AWS Nitro Sandboxes , Amazon EC2 User Guide
- Nitro Sandboxes: Hardware isolation for every EKS pod, ECS task, and EC2 workload , AWS Compute Blog, November 10, 2026
- AWS Nitro Sandboxes: Hardware-enforced workload isolation on Amazon EKS , AWS Containers Blog, November 9, 2026
- Sandboxing on Amazon EKS with isolated and confidential pods , Amazon EKS User Guide
- Run a pod in isolated mode , Amazon EKS User Guide
- Run a pod in confidential mode , Amazon EKS User Guide
- Resource isolation and throttling , Amazon EKS User Guide
- Network isolation , Amazon EKS User Guide
- Considerations , Amazon EKS User Guide
- Create and update a Nitro Sandbox AMI , Amazon EKS User Guide
- Isolated and confidential tasks , Amazon ECS Developer Guide
Third-party material:
- Nitro Sandboxes guide, pull request 6408 , Firecracker, November 9, 2026
- AWS Nitro Sandboxes on EKS vs. Daytona, E2B and Modal , Qovery, November 10, 2026
- What is an AWS Nitro Sandbox, really? , NOFire AI, November 10, 2026
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article