
Agent Sandboxes: Isolating Untrusted Code with a Two-Process Sandbox
A sample, portable design for running model-generated code with defense in depth — a trusted broker process and an unprivileged, network-isolated executor, talking over a Unix-domain socket. A reference to learn from and adapt, not a drop-in production system.
Series: Agent Sandboxes on AWS (5 articles)
- …
- 5Agent Sandboxes: Isolating Untrusted Code with a Two-Process Sandbox This article
Contents
- Introduction
- Two Processes Along a Privilege Boundary
- The Internal Transport: JSON-RPC over a Unix-Domain Socket
- The Executor: the call_tool Inversion
- A Fresh Executor per Run
- Ending a Run Safely
- Rootless Isolation: the User Namespace
- Seccomp: a Syscall Denylist
- Resource Limits: Bounding an Expensive Run
- Bridging the Two Sockets
- The Image
- Validating the Sample
- Conclusion
Introduction
When an AI agent runs code it didn't write — a program a model generated to orchestrate its tools, a notebook cell, a coding agent's edit-run loop — that code is untrusted. It can be accidentally destructive, or deliberately steered (a prompt-injection payload hidden in a tool result or a web page), and it executes on your infrastructure. A strong outer boundary — a VM or a container runtime — is necessary, but it shouldn't be the only boundary: if the language sandbox is escaped, you want the blast radius already contained.
This post walks through a sample design for that: split the sandbox into two OS processes — a trusted broker that owns the outbound connection and any credentials, and an unprivileged executor that runs the untrusted code with no network and a locked-down syscall surface, talking to the broker over a Unix-domain socket. It's the in-VM design that the companion post, Implementing Programmatic Tool Calling on Serverless Lambda MicroVMs , defers to — and the mechanics (Unix-domain sockets, user namespaces, seccomp) are generic Linux and apply to any substrate.
Treat the code here as a reference to adapt, not production-ready: the snippets are trimmed to show the design clearly, and a real deployment needs more around them — thorough error handling, resource limits tuned to your workload, logging and audit, and a test suite against your own threat model. The goal is to make the moving parts and the why behind each one concrete enough to build on.
The surrounding context, in one paragraph: an orchestrator outside the sandbox holds credentials and drives the agent loop. It sends a program to the sandbox to run; the program calls tools via a
call_tool(...) function, and each call is relayed back out to the orchestrator, which makes the real call and returns the result mid-execution. For that programmatic-tool-calling pattern and the restricted-Python execution engine referenced below, see the Agent Sandboxes on AWS pillar and the Secure Agent Sandboxes on EKS deep dive. Here we focus on what happens inside the sandbox.Two Processes Along a Privilege Boundary
Inside the sandbox, the design is split into two OS processes along a privilege boundary — so that untrusted code never shares an address space with the outbound connection or any network access:
- The server (trusted). The entrypoint, holding the connection to the orchestrator — the only channel off the host, and the only process with network access. In the AWS Lambda MicroVMs implementation this is a WebSocket server behind the MicroVM's authenticated endpoint, but the role is substrate-independent: it brokers, it never runs untrusted code.
- The code executor (untrusted). A separate, unprivileged process that runs the model-generated code in a restricted-Python sandbox — allowlisted builtins, blocked imports, captured stdout/stderr (the same engine described in the Amazon EKS deep dive ). It's launched clean for every run: an empty environment, running as an unprivileged user in its own user namespace, holding nothing but one Unix-domain socket back to the server (an inherited file descriptor). The user namespace lets it get its own network namespace (
net=none) with no host privilege, and it runs under a seccomp syscall filter — so even a full escape of the language sandbox has no network to reach and no connection to hijack, its only channel out being that socket.
The split is what contains a language-sandbox escape: the outbound connection and all network access live only in the server, and the executor's only channel out is the socket to that server.

The Internal Transport: JSON-RPC over a Unix-Domain Socket
The two processes talk over a Unix domain socket, using JSON-RPC 2.0 with
Content-Length framing. Using JSON-RPC on this hop mirrors whatever the orchestrator uses on the outer hop, so the server is a near-verbatim relay; Content-Length framing is the same convention LSP and MCP-over-stdio use for JSON-RPC on a raw byte stream. (A UDS works even with the executor in a net=none namespace, because the socket is an inherited file descriptor, not a network endpoint.)Two message shapes carry a run:
| Direction | Message |
|---|---|
| server → executor | {"id":1,"method":"execute","params":{"code":"..."}} |
| executor → server | {"id":101,"method":"call_tool","params":{"tool":"...","args":{...}}} |
| server → executor | {"id":101,"result":{...}} |
| executor → server | {"id":1,"result":{"stdout":"...","status":"completed"}} |
| either | {"id":N,"error":{"code":-32001,"message":"..."}} (app band -32000..-32099) |
The framing is small — a
Content-Length header, a blank line, then the JSON body:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def encode_frame(msg: dict) -> bytes:
body = jsonrpc.encode(msg).encode()
return b"Content-Length: %d\r\n\r\n" % len(body) + body
def read_frame(rfile) -> dict | None: # blocking read; None on clean EOF
headers = []
while True:
line = rfile.readline()
if not line:
return None # EOF
line = line.rstrip(b"\r\n")
if line == b"":
break # blank line -> body follows
headers.append(line)
n = _content_length(headers) # parse; reject absent/negative/over-cap
return jsonrpc.decode(rfile.read(n).decode())The executor uses this blocking reader; the server uses an
asyncio variant that awaits reader.readexactly(n), since it services this socket and the outbound connection concurrently.One detail matters on this hop precisely because the executor is untrusted:
Content-Length is attacker-controlled. An escaped executor can send whatever header bytes it likes, and the server buffers a whole frame before parsing it — so a header claiming a multi-gigabyte body is a memory/denial-of-service lever. Cap it. _content_length rejects an absent, non-numeric, negative, or over-a-max-cap length before the body is read, raising an error the server turns into a terminal response rather than trying to allocate for it:1
2
3
4
5
6
7
8
9
10
11
MAX_FRAME_BYTES = 1 * 1024 * 1024 # one message-size limit, shared with the outer hop
def _content_length(headers) -> int:
for line in headers:
name, sep, value = line.partition(b":")
if sep and name.strip().lower() == b"content-length":
n = int(value.strip()) # ValueError on non-numeric
if n < 0 or n > MAX_FRAME_BYTES:
raise ValueError("bad Content-Length")
return n
raise ValueError("missing Content-Length")Pick one message-size limit and enforce it on both hops — this internal frame cap and the outer connection's maximum message size (whatever your transport's equivalent is; a WebSocket has a
max_size). One number means a message that fits one hop fits the other, with no silently-different ceiling between them where a payload passes the first and is rejected mid-stream by the second. And reject oversized input at the edge: rather than let the transport drop the connection when the model generates a program larger than the limit, have the orchestrator size-check the execute request before sending and return a clean "program too large" error. The cap here is deliberately modest (1 MiB) — comfortably above any real program or tool payload, small enough that a bad frame is cheap to reject.The Executor: the call_tool Inversion
The executor's whole job is a small loop plus the
call_tool it injects into the sandboxed program. call_tool is the interesting part — the inversion: the sandbox initiates a request mid-execution by sending it over the socket and blocking until the matching id comes back.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def make_call_tool(sock, rfile):
def call_tool(payload: dict) -> dict:
req = jsonrpc.request("call_tool", params=payload)
sock.sendall(framing.encode_frame(req))
while True: # block until our reply
msg = framing.read_frame(rfile)
if msg is None:
raise ConnectionError("server closed the socket")
if jsonrpc.is_response(msg) and msg["id"] == req["id"]:
return msg.get("result") or {"error": msg["error"]}
return call_tool
def main(fd: int):
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, fileno=fd)
rfile = sock.makefile("rb")
call_tool = make_call_tool(sock, rfile)
resource_limits.apply() # rlimits: memory / fds / file size
seccomp_filter.install() # lock down before any code
while (msg := framing.read_frame(rfile)) is not None:
if msg.get("method") == "execute":
result = driver_core.execute(msg["params"]["code"], call_tool)
sock.sendall(framing.encode_frame(jsonrpc.success(msg["id"], result)))That gives the lifecycle of one run:
- Receive. The server accepts the orchestrator's
executerequest and forwards the program to the executor as a JSON-RPCexecuteover the UDS. - Run. The executor runs the code in the restricted sandbox. It exposes one function to the running program:
call_tool. - Relay each tool call. The executor has no network of its own, so it never runs tools itself. When the program calls
call_tool(...), the executor sends acall_toolrequest over the UDS and blocks on the reply. The server relays that request to the orchestrator, gets the result, and returns it over the UDS; the executor hands it back into the running program. From the code's point of view,call_toolis just an ordinary function that returns a value. This repeats for every tool call. - Return. When the program finishes, the executor sends the captured stdout and status back over the UDS, and the server forwards it to the orchestrator.
Because the executor is its own process, its side is simple blocking I/O — send a frame, block on the reply — with none of the async plumbing an in-process design would need.
driver_core.execute is the restricted-builtins engine from the EKS deep dive , with call_tool injected as a global; the seccomp_filter.install() call is covered in Seccomp below.A Fresh Executor per Run
The server spawns a fresh executor process for every
execute request — one per run, launched clean and never reused. Runs are one-at-a-time: while a run is in flight the server rejects a second execute rather than multiplexing untrusted runs, and each sequential execute on the same connection gets its own new executor. The server itself, by contrast, lives for the whole session.Launching clean per run is what guarantees no state, file descriptors, or residue from one run's (possibly malicious) code carries into the next — terminate-never-reuse at the process level. In the common per-session pattern you send one program, get the result, and tear the whole box down, so you often see a single executor over its life — but the model supports N sequential runs, each isolated in its own process.
Ending a Run Safely
One subtlety in how the server ends a run: detect the executor's result to build the response, but don't trust it as the termination guarantee. The
execute result (completed or error) is an application-level message from the untrusted process — it tells the server what to forward to the orchestrator, but a run that escaped the language sandbox could report a false completed and keep running, or hang and report nothing at all. So the server drives termination from three independent triggers, only one of which is the self-report:- Result received — forward it to the orchestrator, then actively tear the executor down (close the UDS and reap the process), rather than assuming "it said done" means "it stopped."
- Timeout — a server-side timer is the backstop for a run that never returns; on expiry the server sends
SIGTERMthenSIGKILLand returns a timeout error (-32002). This is the case a thread-based design can't handle cleanly — you can't reliably kill a thread running arbitrary code, but you can always kill a process. - Crash or EOF — if the executor dies on its own (an OOM-kill, or a seccomp violation on an escape attempt), the UDS closes; the server reaps it and returns an error.
- Malformed or oversized frame — a compromised executor can send garbage rather than a valid frame (a bad
Content-Length, an over-cap body, non-JSON). Treat that like a crash: kill the executor and return a protocol error, rather than trusting anything more it sends.
The principle: treat the executor's completion/error as a hint for the response, and the OS (exit status,
waitpid, kill-on-timeout) as the authority for termination. The ability to enforce a hard timeout by killing the process is one of the strongest reasons the executor is a separate process rather than a thread.Whichever trigger fires, the server's rule is that a run always produces exactly one terminal response — a result, or a JSON-RPC error. This is easy to get subtly wrong: if the relay only catches the failures it expects (a clean EOF, a timeout) and an unexpected exception escapes — a parse error on a malformed frame, say — the run task can die silently, leaving the orchestrator waiting forever and, if runs share a connection, wedging the next one behind it. So the relay wraps the whole run and converts any exit — success, timeout, crash, parse error, or an unforeseen exception — into a terminal message that is always sent. A defensive catch-all like this is easy to dismiss as belt-and-suspenders, but on the boundary with an untrusted process it's the difference between a contained failure and a stuck agent.
Rootless Isolation: the User Namespace
To give the executor its own network namespace (
net=none), reach for a user namespace rather than a broad OS-capability grant. Creating a user namespace needs no privilege, and inside it the launcher holds the capabilities required to then create the network namespace — the rootless-container model (as bubblewrap and rootless podman use). This matters because network-namespace creation otherwise requires CAP_SYS_ADMIN, a near-root capability; the default capability set of a locked-down environment won't include it, so the direct route fails until you grant broad capabilities — whereas the user-namespace route needs none.This runs in the child between
fork and exec (a preexec_fn), so the executor is already isolated by the time it starts:1
2
3
4
5
6
7
8
9
CLONE_NEWUSER, CLONE_NEWNET, CLONE_NEWIPC = 0x10000000, 0x40000000, 0x08000000
def harden_child(): # preexec_fn: in the child, pre-exec
uid, gid = os.getuid(), os.getgid()
unshare(CLONE_NEWUSER) # 1. user ns — needs no privilege
write("/proc/self/setgroups", "deny") # 2. single-entry identity map
write("/proc/self/uid_map", f"0 {uid} 1") # (mapping your own id is
write("/proc/self/gid_map", f"0 {gid} 1") # allowed unprivileged)
unshare(CLONE_NEWNET | CLONE_NEWIPC) # 3. now hold caps *in* the usernsunshare here is a small ctypes call into libc (os.unshare only landed in Python 3.12). Run the server itself as an unprivileged user and this maps the executor to a non-root host uid too — no root anywhere, no special capabilities. It does rely on unprivileged user namespaces being enabled in the kernel; if they were disabled, the spawn fails closed rather than running unconfined.Seccomp: a Syscall Denylist
Seccomp is defense in depth on top of
net=none and the dropped privileges — not the primary boundary. Two design choices matter.Apply it from inside the executor, after startup. Install the filter once the interpreter is up and just before you run untrusted code, not in the launcher before
exec. Two reasons, and they reinforce each other. First, the timing: a filter installed pre-exec would have to permit the very execve that starts the executor; installed after startup, it can deny execve outright for the code that follows. Second, the server and executor need different policies, and a seccomp filter binds the process that installs it (and all its children) and can only be tightened, never loosened. The server must keep making the syscalls the executor is denied — creating and using the outbound socket, and fork+execve to launch each executor — so it can't install the executor's restrictive filter without breaking itself. The only way to give the executor a tighter policy than the server is to add it in the executor's own process, after it has split off.Prefer a denylist over an allowlist. A live CPython interpreter issues a large, version- and workload-dependent set of syscalls (
mmap, futex, openat for lazy imports, getrandom, …). An allowlist that misses one kills the interpreter. Default to allow, and block the syscalls untrusted code has no legitimate reason to make:1
2
3
4
5
6
7
8
9
10
11
12
13
14
import pyseccomp as seccomp # libseccomp bindings
KILL = ["execve", "execveat", "fork", "vfork", "clone", "clone3", "ptrace",
"mount", "umount2", "unshare", "setns", "bpf", "init_module",
"kexec_load", "keyctl", "perf_event_open"] # never legit here
EPERM = ["socket", "socketpair", "connect", "bind", "listen", "accept"] # new sockets
def install():
f = seccomp.SyscallFilter(defaction=seccomp.ALLOW) # default-allow
for name in KILL:
f.add_rule(seccomp.KILL_PROCESS, name)
for name in EPERM:
f.add_rule(seccomp.ERRNO(errno.EPERM), name)
f.load() # libseccomp sets NO_NEW_PRIVS, so this works unprivilegedSend/recv stay allowed so the executor's inherited UDS keeps working; only creating or connecting new sockets is blocked. Guard each
add_rule if you target multiple architectures (a syscall may not exist everywhere), and make the whole thing a no-op when the bindings aren't present so the image still runs.Resource Limits: Bounding an Expensive Run
Isolation stops untrusted code from reaching things; it doesn't stop it from consuming things. A program that allocates until the box is out of memory, spins forever, forks a bomb, fills the disk, or prints without bound can degrade or take down the sandbox — and if the memory pressure kills the server rather than the executor, it takes the whole session with it. A few cheap, rootless controls bound each of those:
- CPU / wall-clock — the kill-on-timeout from Ending a Run Safely already handles a runaway loop.
- Processes / threads — the seccomp denylist already blocks
fork/clone, so a fork bomb never starts. - Memory, disk, file descriptors — POSIX rlimits, set by the executor just before it runs code (the same "harden after startup, before the code" placement as seccomp):
1
2
3
4
5
6
7
import resource # MB = 1024 * 1024
def apply(): # in the executor, before untrusted code
_set(resource.RLIMIT_AS, 1024 * MB) # ~1 GiB virtual memory — a backstop
_set(resource.RLIMIT_FSIZE, 16 * MB) # max file size — caps a disk fill
_set(resource.RLIMIT_NOFILE, 64) # max open file descriptors
_set(resource.RLIMIT_CORE, 0) # no core dumpsRLIMIT_AS is the memory backstop, but it's blunt: it caps virtual address space, which a 64-bit interpreter reserves far more of than it ever makes resident, so you must set it generously or you'll kill legitimate programs. For an accurate ceiling, add a server-side RSS watchdog — the trusted server already supervises the executor, so it polls the executor's real resident memory and kills it if it crosses the budget:1
2
3
4
5
6
async def rss_watchdog(proc, max_rss_mb): # runs alongside the relay
while proc.returncode is None:
await asyncio.sleep(0.5)
if _rss_mb(proc.pid) > max_rss_mb: # from /proc/<pid>/status
proc.kill() # -> UDS EOF -> terminal memory error
returnThe pairing is deliberate: the watchdog gives the tight, RSS-accurate limit with no privilege, and the rlimit sits under it as a kernel-enforced backstop. Together they're the rootless stand-in for a cgroup
memory.max — cgroups would be more accurate still (and add CPU-quota and PID caps), but delegating a cgroup subtree needs a privileged step at boot, which trades away the no-root property this design worked to keep; for a rootless sample, watchdog-plus-rlimits is the better fit.One more consumable is output. Untrusted code that prints without bound balloons the executor's memory as the server buffers the result, and blows past the message-size limit on the way back. Cap captured stdout/stderr and truncate gracefully, so a flood becomes a large-but-bounded result the run still completes:
1
2
3
4
5
6
class CappedBuffer: # stands in for io.StringIO
def write(self, s):
if self._size < self._limit: # keep up to the cap ...
...
else:
self._dropped += len(s) # ... and just count the restTwo traps worth calling out.
RLIMIT_NPROC looks like the natural PID-bomb limit, but it's enforced per real uid across the whole system, and the server shares the executor's uid — so a low value starves the server. Lean on seccomp for process creation instead. And keep the output cap comfortably below the message-size limit, so a completed result always fits with JSON-envelope headroom.Bridging the Two Sockets
On the server side there's one deadlock to avoid: it relays between the UDS (to the executor) and the outbound connection (to the orchestrator), and it must service both at once. If it blocks waiting for a tool-call result on the outbound side while the executor is trying to send it another frame, the
call_tool round-trip deadlocks. Run the two directions concurrently (async tasks, or two threads). The executor side needs no such care: as its own process, it just blocks on the socket between calls.The Image
The image is a standard Dockerfile — install Python (and
libseccomp for the filter), add an unprivileged user, add the sandbox code (the JSON-RPC helpers, the executor, and the server), and start the server as the entrypoint. Run everything as the unprivileged user: the server needs no privilege, because the executor isolates itself with a user namespace rather than host capabilities.1
2
3
4
5
6
7
8
9
FROM <your-base-image>
RUN <install> python3 python3-pip libseccomp
RUN useradd --system --no-create-home --shell /sbin/nologin sandbox
WORKDIR /app
COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt # websockets + pyseccomp (pure-Python)
COPY *.py /app/
USER sandbox
CMD ["python3", "server.py"]No capability grant is needed — the user-namespace approach requires none. (On Lambda MicroVMs specifically, this means the image is built with no
additionalOsCapabilities; the companion post covers that platform detail.)Validating the Sample
A design like this is only as good as the evidence that its boundaries actually hold, so it's worth exercising them adversarially — and doing so is what surfaced the frame-size guard and the always-return-a-terminal rule above. Two small, throwaway harnesses go a long way; neither needs to be elaborate.
Fuzz the frame parser. The parser sits on the trusted side of an untrusted hop, so it's attacker-facing: a compromised executor can send it arbitrary bytes. Feed it crafted and random garbage — truncated frames, non-numeric and negative and enormous
Content-Length values, invalid UTF-8, non-JSON bodies — and assert it always fails safe: returns a message, returns None (EOF), or raises a catchable error, but never crashes unexpectedly, hangs, or tries to allocate unboundedly. This runs in-process with no infrastructure, which is exactly why it's cheap to keep around.Probe containment end to end. Against a live sandbox, run the payloads a real attacker would, through the ordinary
execute path. The revealing ones deliberately bypass the language sandbox with a known escape and then attempt the thing the OS layers must stop — spawning a program (blocked by seccomp), reaching the network (net=none), reading credentials from the environment (there are none) — plus a runaway loop that must hit the kill-on-timeout. The language-sandbox escape is expected to succeed; the point is that every payload is still contained by the layer behind it, which is what "defense in depth" has to mean concretely. A companion check runs a crashing program and then a benign one on the same connection, confirming the crash returns a terminal error and the connection still serves the next run.None of this proves the sample secure — it isn't a substitute for review against your own threat model — but it turns "we think it's isolated" into "here's what we tried and what held," and it's how the rough edges above got found.
Conclusion
The design is small but demonstrates real defense in depth: a trusted broker that owns the connection and credentials, and an untrusted executor that runs each program in a fresh, unprivileged, network-isolated, syscall-filtered process — with the outer VM or container as the last boundary rather than the only one. Every piece here — the Unix-domain socket, the user namespace, the seccomp filter, the kill-on-timeout — is generic Linux, so the sample ports across sandbox substrates.
Again, treat it as a starting point rather than a finished product: tune the resource limits and seccomp policy to your own workload, add observability and audit logging, harden the remaining error paths, and test against your own threat model before relying on it. What's worth taking away is the shape — the privilege split and the reasoning behind each control.
For where this runs as a serverless per-session sandbox with built-in lifecycle management, see the companion post, Implementing Programmatic Tool Calling on Serverless Lambda MicroVMs . For the concepts and the full landscape, start with the Agent Sandboxes on AWS pillar; for the Kubernetes path with three isolation runtimes, see Secure Agent Sandboxes on EKS .
Series: Agent Sandboxes on AWS (5 articles)
- …
- 5Agent Sandboxes: Isolating Untrusted Code with a Two-Process Sandbox This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article