A coding agent on AWS services only
What happens to the seven layers when five of them are managed
See it run. Agent Loop Trace is a live animation of the loop this article describes: one request enters, three Converse calls and two tool calls happen, the response streams back — and the input-token meter shows the cost the loop hides.
The claim under test
Strip the UI off a coding agent and what is underneath is a while loop: ask the model, take the result, decide whether it wants a tool, run the tool, ask again. That is true, and it is why building one from scratch is a good way to learn how every other agent works.
This article asks the narrower question: if the loop is the easy part, which parts are hard, and what happens to those parts when you are only allowed AWS services?
The answer is specific. A from-scratch agent has seven layers. On AWS, five of them are managed services. And the two failure modes that hand-written agent infrastructure reliably produces are both in layers that stay hand-written.
1. The seven layers
This is the decomposition a careful from-scratch implementation arrives at, and it is a good one:
| layer | what it does | what writing it costs |
|---|---|---|
| model adapter | flatten provider differences behind one complete() call | ~500 lines per provider: JSON shaping, HTTP, SSE parsing, error mapping |
| agent loop | ask → tool calls? → execute → append → repeat | ~50 lines. The easy part. |
| tool registry | name → handler, plus JSON Schema for the model | a map, plus schema plumbing |
| session | conversation memory that survives a restart | append-only log, an index, tree links for fork and rollback |
| compaction | keep the context window from overflowing | a token estimate, a threshold, a summarisation call |
| resources | load project rules and skill files, inject them into the prompt | frontmatter parsing, precedence rules |
| event bus + plugins | let outsiders observe and intercept the loop | dispatch machinery, a plugin host |
| transport | expose the engine to any client | a socket, a framing protocol, an event vocabulary |
Splitting model adapter / kernel / product is the right decomposition, and putting the engine behind a socket so the client can be written in any language is a better decision than most production agents make.
2. The two failure modes
Two defects show up again and again in hand-built agents, and they are worth naming before the architecture, because the architecture is the argument about where they come from.
A. The tool definitions never reach the model. The registry holds six tools. The request builder forgets to serialise them into the request body. Nothing fails: the model simply answers as a chat model, never asking for a tool, and the agent quietly stops being an agent. There is no error to see, because an empty tool list is a valid request.
B. The plugin host has no isolation. Plugins are loaded into the agent process to observe and intercept the loop. A hook that returns "block this tool call" is a function call; a hook that instead reads
~/.aws/credentials is also just a function call. There is no boundary to enforce, because plugin and host share an address space.Neither is a mistake in the loop. Both are in hand-written infrastructure. Hold on to them; §3.1 and §8.3 return to each.
3. Layer 1 — the model adapter becomes one API call
Flattening two providers costs about a thousand lines, and the differences are real: one sends
messages[] and the other content[]; tool calls are tool_calls[] in one and a tool_use block in the other; streaming is data: lines versus event: lines.On AWS this layer is Amazon Bedrock's Converse API [2].
Converse and ConverseStream present one request and response shape across the models Bedrock hosts, with tool use as part of the contract rather than a per-provider dialect.1
2
3
4
5
6
7
8
# The whole of the model adapter layer, as an API call.
resp = bedrock.converse(
modelId=model_id,
system=[{"text": system_prompt}],
messages=messages, # one shape, every model
toolConfig={"tools": tool_specs}, # typed; see §3.1
inferenceConfig={"maxTokens": 4096, "temperature": 0.2},
)
The response carries
stopReason, and the loop's entire branch condition is whether it equals tool_use. Tool results go back as a toolResult block. Streaming is the same call with converse_stream, yielding contentBlockDelta events — the streaming callback, without the SSE parser.What you give up. Bedrock hosts a specific catalogue of models. A hand-written adapter can point at anything with an HTTP endpoint, including a model on a laptop. That flexibility is real and the AWS-only constraint costs it. If the requirement is "any OpenAI-compatible endpoint", this layer does not go away and §12 applies.
3.1 Why failure mode A cannot happen here
The bug is that the tools were never serialised, so the model silently ran with none.
That bug class needs a hand-rolled serialiser to live in.
toolConfig is a typed parameter of a published API: omit it and no tools are sent — visibly, because it is absent from the call you wrote — and malformed input returns a ValidationException rather than an empty tool list. There is no path where the field is present in your data model, absent from the wire, and silent about it.This is the thesis in miniature. The managed service did not fix the bug. It removed the place where the bug could live.
4. Layer 2 — the loop
The loop is worth keeping. It is thirty lines and it is not the hard part.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
def run_turn(session_id, user_input, tool_specs, max_turns=20):
messages = load_history(session_id) + [user_msg(user_input)]
for turn in range(max_turns): # the safety valve, kept
resp = bedrock.converse(
modelId=MODEL, messages=messages,
toolConfig={"tools": tool_specs},
system=[{"text": SYSTEM_PROMPT}],
)
messages.append(resp["output"]["message"])
if resp["stopReason"] != "tool_use":
return resp, messages
results = []
for block in resp["output"]["message"]["content"]:
if "toolUse" not in block:
continue
tu = block["toolUse"]
out = invoke_tool(tu["name"], tu["input"]) # §5
results.append({"toolResult": {
"toolUseId": tu["toolUseId"], # correlation is mandatory
"content": [{"json": out}],
"status": "success" if out.get("ok") else "error",
}})
messages.append({"role": "user", "content": results})
raise MaxTurnsExceeded(max_turns)
max_turns stays, for the reason every implementation arrives at: a model can loop on tools forever, and the ceiling is a safety valve rather than a tuning parameter.Three choices about where it runs:
| host | when |
|---|---|
| AWS Lambda | a turn that finishes inside the function timeout. One function, one turn. |
| AWS Step Functions [5] | the loop as a state machine, with Retry and Catch per error class — retry classification declared rather than coded — and no timeout ceiling on the loop itself. |
| AgentCore Runtime [1] | long-running or multi-hour sessions with per-session isolation, which is what a coding agent actually is. |
Hand-written loops classify errors into retryable and not. Step Functions expresses that as a
Retry array keyed on error name, with backoff and jitter as parameters rather than a hand-written sleep. The classification is still yours; the machinery is not.5. Layer 3 — tools, and the sandbox question
The AWS shape is one Lambda function per tool, with AgentCore Gateway [1] in front to present them as MCP tools with runtime discovery — so a tool added to the account appears to the agent without a redeploy.
| tool | AWS |
|---|---|
| read_file, write_file | Lambda over S3 or EFS, scoped by execution role |
| bash | AgentCore Code Interpreter — an isolated session, see §5.1 |
| (none in most hand-built agents) | AgentCore Browser — managed browser instances |
| search, list | Lambda; or OpenSearch behind a Lambda |
5.1 A command denylist is not a security boundary
The standard first attempt at making
bash safe is an interceptor that pattern- matches the command. It usually looks like this:1
2
3
4
5
6
7
8
9
-- the shape this takes in almost every hand-built agent
on_tool_start = function(event)
if event.tool_name == "bash" then
if event.arguments:find("rm -rf", 1, true) then -- plain substring
return { action = "block", reason = "dangerous command" }
end
return { action = "modify", arguments = "set -e; " .. event.arguments }
end
end
A plain substring search blocks the literal eight characters
rm -rf and nothing else. Every one of these passes:1
2
3
4
5
6
rm -fr / # flags reversed
rm --recursive --force / # long options
rm -rf / # two spaces
rm -r -f / # flags split
R=rm; $R -rf / # indirection through a variable
find / -delete # a different program entirely
And prepending
set -e; changes error-exit semantics while adding no safety.The point is not that this particular filter is weak. It is that a denylist over an unbounded input language cannot be made strong. The command language is Turing-complete; the filter is a string comparison. The filter loses, always.
The AWS answer is not a better pattern. It is to stop trying to decide whether the command is dangerous and make the question uninteresting:
- the command runs in an AgentCore Code Interpreter session, isolated by construction;
- the session's role reaches exactly the S3 prefix for that workspace and nothing else;
- the filesystem is discarded when the session ends.
rm -rf / inside that boundary destroys a sandbox that was going to be thrown away. The blast radius is bounded by an IAM policy — a closed enumeration of what is allowed — instead of by a denylist of what is forbidden, which can never be complete.Bedrock Guardrails has a place, but a different one: content and topic policy on the model's input and output, not command filtering. Use it for what it is, and do not let its presence imply that the shell is being policed.
6. Layer 4 — sessions
A good hand-written session format is append-only JSON Lines, one object per line, each entry carrying a
parent_id so the conversation is a tree rather than a list — which is what makes fork and rollback possible — with corrupt lines skipped on load rather than crashing it.The AWS translation keeps the structure and changes the storage:
| # | access pattern | PK | SK |
|---|---|---|---|
| 1 | replay a session in order | SESS#<session_id> | ENTRY#<zero-padded seq> |
| 2 | session header | SESS#<session_id> | META |
| 3 | children of an entry (fork) | GSI: parent_id | ENTRY#<seq> |
| 4 | a user's sessions, newest first | GSI: USER#<sub> | <created_at desc> |
Pattern 1 is a single-partition
Query returning items in sort-key order, so "replay the session" is one call with native pagination. Pattern 3 is the tree: one GSI turns parent_id from a field you scan for into a key you query. Large tool outputs go to S3 with the item holding a pointer, because a DynamoDB item has a size limit and a bash transcript will exceed it.Or skip the design entirely: AgentCore Memory [1] manages session and long-term memory as a service, including carrying context between sessions — the feature that separates an agent you use once from one that learns your codebase.
The trade is the usual one. DynamoDB is yours: schema, retention, export. AgentCore Memory is less code and less control. For a coding agent whose session history is the user's source code interleaved with their questions, "where does this live and who can read it" is worth keeping in your own hands, and §12 does not pretend otherwise.
7. Layer 5 — compaction, and an estimator that is wrong where it matters
Hand-written compaction needs to know how full the context is, and the usual shortcut avoids a tokenizer:
1
2
/* the standard approximation */
size_t estimate_tokens(const char *text, size_t len) { return len / 4; }
Four characters per token is a reasonable rule of thumb for English. For CJK text it is wrong by roughly a factor of two to three — Chinese, Japanese and Korean run closer to one token per 1–1.7 characters, because each character typically costs its own token rather than amortising across a word.
The direction of the error is the dangerous one.
len / 4 under-counts CJK, so the "do we need to compact?" check returns false when the real context is already two to three times the estimate. Compaction fires too late, and the failure mode is the context overflow that compaction exists to prevent. For an agent whose users write Chinese prompts about Chinese-commented code, the estimator is least accurate exactly where it is most load-bearing.On AWS the heuristic is unnecessary, because the count is already in the response [2]:
1
2
3
4
resp = bedrock.converse(...)
usage = resp["usage"] # inputTokens, outputTokens, totalTokens
if usage["inputTokens"] > COMPACT_THRESHOLD:
compact(session_id)
Bedrock returns the model's own token accounting per call. There is no estimator to be wrong, in any language. Same shape as §3.1: not a better heuristic, but the removal of the need for one.
Compaction itself stays a design decision — what to summarise, what to keep verbatim, whether to preserve tool results. Only the trigger becomes exact.
8. Layers 6 and 7 — events and plugins
8.1 Do not put every token on a bus
The obvious mapping for an event bus is EventBridge, and it is half right.
| event | transport | why |
|---|---|---|
| turn_start, tool_start, tool_end, turn_end, result | EventBridge | coarse lifecycle; other systems may subscribe; a few per turn |
| delta (per token) | API Gateway WebSocket [4], direct | one per token, latency-critical, exactly one consumer |
| error | both | the client needs it now; operations need it durably |
Putting per-token deltas through an event bus is the mistake this table exists to prevent. A thousand-token response becomes a thousand bus events, each with per-event cost and delivery latency, to reach the one browser waiting for them. EventBridge is for fan-out to systems that do not exist yet; a token stream has exactly one reader and needs to arrive immediately.
8.2 The transport
A raw TCP socket with JSON-lines framing is the smallest thing that works, and documenting the protocol so a client can be written in any language is the right instinct. The AWS equivalent is an API Gateway WebSocket API [4] — full-duplex, streaming, and it brings Cognito authorisation and per-connection identity that a raw socket has to invent.
A ping/pong event type turns out to be load-bearing rather than decorative: WebSocket APIs have documented quotas on idle timeout, total connection duration and frame size, and a coding-agent turn that spends four minutes in a build will hit the idle timeout without a keepalive. Check the current quotas before designing around long-lived connections; a turn longer than the maximum connection duration needs the client to reconnect and resume by
session_id, which §6's session design already supports.8.3 Why IAM is the answer to failure mode B
An embedded scripting runtime shares an address space with its host, so a "blocking" hook is a function call and nothing stops that function from doing something else. That is not a missing feature; it is the architecture.
On AWS a plugin is a Lambda function with its own execution role:
1
2
3
┌── plugin-a (Lambda) role: read session metadata only
EventBridge rule ───┼── plugin-b (Lambda) role: write to one S3 prefix
on tool_start └── plugin-c (Lambda) role: nothing at all
Isolation is not a feature someone remembered to add. It is what a separate execution environment and a separate IAM role are. A plugin that should not read the session store is given a role without that permission, and "can this plugin exfiltrate my credentials" has an auditable answer.
The cost, stated honestly. An in-process hook returns in microseconds. A Lambda invocation costs milliseconds and can cold-start. For a blocking interceptor — one whose verdict gates the tool call — that latency is on the critical path of every tool use. Three options, and the choice is real:
- Blocking, synchronous invoke. Correct interception, pay the latency.
- Non-blocking, EventBridge. No latency on the critical path, but the plugin observes rather than intercepts — it cannot block.
- Policy instead of plugin. Express the rule as an IAM policy or an AgentCore Policy control, enforced at the boundary with no invocation.
Option 3 is the one to reach for first, and it is §5.1 generalised: most "blocking" plugins are denylists that a policy expresses better and a sandbox makes unnecessary.
9. The whole thing
The live trace animates one request through this diagram.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
DEVELOPER ┌───────────────── AWS Cloud ──────────────────┐
│ │ │
│ 1 WebSocket │ ┌────────────────┐ │
├────────────────────────────▶│ │ API Gateway │◀── Cognito (identity) │
│ ◀───────────────────────── │ │ WebSocket API │ │
│ deltas, direct │ └───────┬────────┘ │
│ │ 2 │ │
┌──┴───┐ │ ▼ │
│ TUI │ │ ┌────────────────────────┐ │
│ or │ │ │ Agent loop │ §4 │
│ IDE │ │ │ Lambda / Step Functions│ │
└──────┘ │ │ / AgentCore Runtime │ │
│ └──┬──────────┬──────┬───┘ │
│ 3 │ 4 │ 5 │ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌──────┐ ┌──────────────┐ │
│ │ Bedrock │ │ Agent│ │ DynamoDB │ │
│ │ Converse│ │ Core │ │ sessions §6 │ │
│ │ §3 │ │Gate- │ │ S3 (blobs, │ │
│ │ │ │ way │ │ skills §10) │ │
│ └─────────┘ └───┬──┘ └──────────────┘ │
│ │ 6 │
│ ┌─────────┼──────────┐ │
│ ▼ ▼ ▼ │
│ ┌────────┐ ┌────────┐ ┌────────────┐ │
│ │ Lambda │ │ Agent │ │ AgentCore │ │
│ │ tools │ │ Core │ │ Browser │ │
│ │ r/w │ │ Code │ │ │ │
│ │ files │ │ Interp.│ │ │ │
│ └────────┘ └────────┘ └────────────┘ │
│ §5, the real sandbox │
│ │
│ EventBridge ──▶ plugin Lambdas (own roles) │
│ §8.3 │
│ AgentCore Observability + CloudWatch + X-Ray │
└──────────────────────────────────────────────┘
| hand-written layer | AWS service | what you stop writing |
|---|---|---|
| model adapter | Amazon Bedrock Converse / ConverseStream | ~1,000 lines of provider flattening |
| agent loop | Lambda, Step Functions, or AgentCore Runtime | ~0 — keep it, it is the easy part |
| tool registry | Lambda + AgentCore Gateway (MCP, runtime discovery) | registry and schema plumbing |
| bash sandbox | AgentCore Code Interpreter | the isolation §5.1 says you need |
| — | AgentCore Browser | a capability most hand-built agents lack |
| session | DynamoDB + S3, or AgentCore Memory | log writer, index, tree links |
| compaction trigger | Bedrock usage in the response | the estimator, and its CJK error (§7) |
| resources | S3 + DynamoDB | frontmatter loader (mostly stays, §10) |
| event bus | EventBridge (lifecycle) + WebSocket (deltas) | dispatch machinery |
| plugins | Lambda per plugin, IAM-scoped | the isolation §8.3 says you need |
| transport | API Gateway WebSocket API | socket handling, framing, auth |
| third-party identity | AgentCore Identity | OAuth token handling |
| observability | AgentCore Observability, CloudWatch, X-Ray | tracing |
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article