
Beyond Retries and Timeouts: Circuit Breakers for Agentic AI Workflows
This article looks at why circuit breakers, a pattern built for deterministic, code-driven callers, need to be rethought for AI agents that reason around failures instead of backing off. It walks through where the breaker needs to sit, how to track state correctly, and how to respond in a way a language model won't just retry past.
You've spent a decade building circuit breakers into service meshes. You know the pattern cold: a downstream service starts failing, the breaker trips, calls stop going out, the system degrades gracefully instead of falling over. It's one of those patterns that's so second nature you barely think about it anymore — you just build it in, the way you'd add a health check or a timeout.
Then you deploy your first production agent, its retrieval tool starts timing out at 2am, and instead of failing gracefully, the agent just keeps trying. It rephrases the query. It tries a different tool. It tries the same tool again. Twelve minutes and 40 tool calls later, you have a five-figure Bedrock bill and no incident alert, because nothing "crashed." The agent was doing exactly what it was built to do: reason its way around a problem.
That's the problem. Not a bug, not a misconfiguration — the pattern you've relied on for years quietly assumed something about the caller that an agent doesn't satisfy anymore. This piece is about where that assumption breaks, and where a circuit breaker actually needs to live once the thing calling your tools is a model instead of code.
Why the old pattern doesn't carry over cleanly
Circuit breakers, retries, timeouts — all of it rests on one assumption we've built distributed systems around for years: the caller is deterministic. It's code. When a Step Functions state calls a Lambda that hits a flaky API, you know exactly where that call sits — state 3 of 7 — and you can wrap it with a retry policy or a breaker, because the shape of the workflow is fixed before it ever runs. That's the whole idea behind decoupling a pipeline with EventBridge and Step Functions: you can reason about failure because you can list out the steps ahead of time.
An agent breaks that at the root. The "step" isn't a state in a graph you designed — it's a decision the model makes on the fly, based on whatever came back from the last tool call. You don't control whether it retries a failing tool, or how many times. That retry behavior isn't a policy you set; it's just what happens when a model tries to be helpful and doesn't have a strong reason to stop.
Which brings up the real question: if you can't put a breaker "between step 3 and step 4" because there's no step 3, where does resilience even go?
Where the breaker actually needs to sit
It has to move below the agent, into a layer the model doesn't get a say in. You can't fix this by telling the model in a prompt not to retry too much — that's a suggestion, not a control, and it's exactly the kind of thing that falls apart under pressure, right when you need it most. The breaker needs to sit between the agent's decision and the actual tool call, so it holds no matter what the model decides to try next.
Architecturally, that's the tool-gateway layer — Amazon Bedrock AgentCore Gateway, API Gateway + Lambda, or a hand-rolled Lambda in front of your tools (increasingly, these are exposed as MCP servers rather than plain APIs, but the placement is the same either way). The point is the agent never talks to the real downstream API directly. It talks to a broker, and the breaker state lives in that broker — not in the agent's context window, and definitely not in the model's judgment.
The mechanics
The state machine itself is one you've built before. AWS's own Prescriptive Guidance on the circuit breaker pattern walks through the same three states, using a Step Functions workflow that checks circuit status in DynamoDB before calling a downstream service, and fails immediately if the circuit's open. It's worth a look — it's essentially the non-agentic version of what we're building here, same shape, just a different kind of caller on the other end. Three states, tracked per tool (or per tool and agent, if you want blast radius scoped to a single session instead of shared across everyone hitting that tool):
- CLOSED — calls go through normally. Failures increment a counter within a rolling time window; N failures inside that window trips the breaker.
- OPEN — calls get rejected before the downstream call is even attempted, for a cooldown period.
- HALF_OPEN — after cooldown, one trial call is let through. If it succeeds, close the circuit. If it fails, back to OPEN, optionally with a longer cooldown next time.
That scoping decision — per tool, or per tool and agent — is worth pausing on, because it's a real tradeoff and not just an implementation detail. A breaker scoped globally per tool means one flaky CRM integration takes that tool offline for every agent and every session hitting it — which is exactly what you want if the tool is genuinely down, but it's a big blast radius if the failures are actually specific to one bad request. Scope the breaker per session instead, and a struggling conversation gets contained to itself while everyone else keeps working — at the cost of every session having to independently rediscover that the tool is down, which is slower to detect and noisier to observe. There's no universally right answer here; it comes down to whether the tools behind your agents tend to fail as a whole, or fail in ways tied to specific inputs.
State lives in DynamoDB, keyed by tool name (or by tool and session, depending on which scoping you land on above). This is the piece that runs inside the broker Lambda we talked about above — it's what actually tracks failures and flips the circuit for a given tool, before an agent's call to it ever reaches the real downstream API.
Two details matter here. DynamoDB's TTL deletion can lag by hours, so it's not something breaker state should depend on — TTL is just for cleaning up old records, while the actual open/closed decision comes from comparing
opened_at and cooldown_until against the current time. And the failure counter has to reset once its window passes, or a handful of unrelated failures scattered across a normal day will eventually add up and trip the breaker for no real reason.import boto3import time
table = boto3.resource("dynamodb").Table("ToolCircuitState")
def record_failure(tool_name, threshold=5, window_seconds=60, cooldown_seconds=120): now = int(time.time()) item = table.get_item(Key={"tool_name": tool_name}).get("Item", {})
window_expires_at = item.get("window_expires_at", 0) if now > window_expires_at: # window has lapsed - start a fresh count instead of accumulating forever failure_count = 1 window_expires_at = now + window_seconds else: failure_count = item.get("failure_count", 0) + 1
should_open = failure_count >= threshold and item.get("state") != "OPEN"
table.update_item( Key={"tool_name": tool_name}, UpdateExpression=( "SET failure_count = :count, " "window_expires_at = :expiry, " "ttl = :cleanup_ttl" + (", #s = :open, opened_at = :now, cooldown_until = :cooldown" if should_open else "") ), ExpressionAttributeNames={"#s": "state"} if should_open else {}, ExpressionAttributeValues={ ":count": failure_count, ":expiry": window_expires_at, ":cleanup_ttl": now + max(window_seconds, cooldown_seconds) * 4, **({":open": "OPEN", ":now": now, ":cooldown": now + cooldown_seconds} if should_open else {}), }, )
def is_circuit_open(tool_name): item = table.get_item(Key={"tool_name": tool_name}).get("Item", {}) if item.get("state") != "OPEN": return False # state check is timestamp-driven, not TTL-driven return int(time.time()) < item.get("cooldown_until", 0)None of that is new — it's a standard breaker, just running serverless. The part that's actually different for agents is what you hand back when the circuit is open, because the caller here isn't code that catches an exception and moves on. It's a model that's going to read whatever you give it and act on it.
What a normal breaker doesn't know how to say
If you send back a generic error — a timeout, a 503, an empty payload — the model treats it like any other transient failure and does what models do with transient failures: tries again, or tries something close enough. That's the exact loop from the 2am story. The rejection has to be written as an instruction the model can't talk itself past, not a status code it gets to interpret:
def open_circuit_response(tool_name, cooldown_until): return { "toolResult": { "status": "unavailable", "error_type": "CIRCUIT_OPEN", "retryable": False, "tool_available_after": cooldown_until, "message": ( f"'{tool_name}' is temporarily disabled after repeated failures " "and won't accept calls for the next few minutes. " "Do not retry this tool or attempt an equivalent call. " "If a fallback tool is available, use that instead. " "Otherwise, tell the user this capability is temporarily unavailable." ), } }That plain "do not retry" is doing the actual work here — not the rejection itself. But models don't always follow instructions embedded in a message reliably, so it's worth not putting all the weight on that one sentence. The structured fields around it —
error_type , tool_available_after — matter just as much, because they let anything sitting above the model (an orchestrator, a supervising agent, your own retry logic) make the same "don't retry" decision without having to parse natural language to get there. A breaker that blocks the call but hands back an ambiguous error hasn't really solved anything for an agent; it's just pushed the retry storm up one layer, into the model's next decision.
Making the outage visible
The other reason that 2am incident goes twelve minutes without anyone noticing isn't just the retries — it's that nothing pages, because no request technically failed at the infrastructure level. The fix is to treat circuit state changes as real events instead of something buried inside a Lambda:
import boto3, json
eventbridge = boto3.client("events")
def emit_circuit_event(tool_name, new_state, failure_count): eventbridge.put_events(Entries=[{ "Source": "agent.circuit-breaker", "DetailType": f"Circuit{new_state.title()}", "Detail": json.dumps({ "tool": tool_name, "state": new_state, "failure_count": failure_count, }), "EventBusName": "agent-observability-bus", }])Point an EventBridge rule off
CircuitOpened at whatever pages your team, and the story from the top changes completely — instead of a quiet cost leak you find the next morning, you get an alert the moment the circuit trips, before the model gets forty more tries at it. Worth emitting CircuitClosed too, not just the open event — recovery is just as much a signal as failure, and knowing a tool came back on its own (versus needing someone to force it) is useful history to have sitting in the same event stream.
What actually changed
The breaker pattern itself hasn't changed. What changed is where it has to live and what it has to say. A normal circuit breaker protects a shaky downstream dependency from load generated by predictable, code-driven callers. An agentic one is doing a second job: protecting you from a caller that will keep improvising around a failure for as long as you let it, and won't stop just because something went wrong once.
The Bedrock bill from the story at the top is the easiest cost to point at, but it's not the only one worth worrying about. Forty consecutive tool calls also mean forty times the latency stacking up in front of the user, forty times the tokens burned on the same dead end, and a response that arrives minutes later than it should — if it arrives at all. And if that tool happens to be a shared dependency, every other agent hitting it inherits the same problem at the same time. Cost is just the number that shows up on a dashboard; the rest of it shows up as a worse product.
Same three states, same failure domain — just moved down a layer, and rewritten for a caller that talks back.
Final thoughts
This is one pattern, applied to one failure mode, in a space that's still being figured out. Agentic systems are going to keep surfacing these kinds of gaps — places where a pattern we've trusted for years quietly assumed something about the caller that no longer holds. Circuit breakers are just the one that happened to be sitting right in front of me. There's probably a similar rethink waiting for bulkheads, for backpressure, for rate limiting — anywhere we've built resilience around the assumption that the thing on the other end of the call is code. Worth pulling on that thread and seeing where else it leads.
Disclaimer: Please note that AWS is constantly evolving, and new features may be available since the release of this blog post. It's recommended to review the latest documentation to determine the most suitable solutions for your specific needs. This blog is a reference guide only. Additionally, ensure that the proposed solutions comply with your organization's security and compliance requirements, as some services may be relatively new and may not be fully compliant with all industry standards.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article