
One AI Agent or Five? When Multi-Agent Architecture Becomes a Liability
Learn when multi-agent AI systems add unnecessary cost, latency and complexity, and how to choose between one agent and several on AWS Bedrock.
Series: My Cloud Experience (3 articles)
- 1One AI Agent or Five? When Multi-Agent Architecture Becomes a Liability This article
A customer asks an AI assistant to check an order, explain a refund, and update the delivery address. One agent could retrieve the order, check the payment record, validate the address change, and respond. Or the system could route the request through a supervisor, an order agent, a payment agent, a shipping agent, and a response agent.
Both designs might work. The second one also creates more conversations, more decisions, and more opportunities for something to go wrong.
Multi-agent systems are attractive because they resemble a team of specialists. Each agent gets a defined responsibility, its own instructions, and a smaller collection of tools. That sounds easier to maintain than one assistant that knows about everything.
The problem starts when we confuse dividing responsibilities with improving the application. Five agents are not automatically more accurate than one, and the extra coordination can become the most expensive part of the request.
Start With the Work, Not the Number of Agents
Imagine an internal support assistant running on AWS. It needs to answer account questions, retrieve invoices, check order status, and occasionally create a support ticket.
A straightforward architecture could use one agent with four carefully defined tools.
User | v Single Agent | +-----------+-----------+ | | | v v v Account API Billing API Order API | v Ticket APINow consider the multi-agent version.
User | v Supervisor Agent | +-----------+-----------+ | | | v v v Account Agent Billing Agent Order Agent | | | v v v Account API Billing API Order API | v Response AgentThe second design introduces new boundaries. The supervisor must identify the right collaborator, pass sufficient context, interpret its result, and decide whether another collaborator is needed.
Those steps may be justified if each specialist requires different permissions, knowledge, or independent ownership. If the specialists simply wrap APIs that the original agent could call directly, the additional agents may be doing little more than forwarding requests.
Before creating another agent, ask what it will own that cannot be represented as a tool, a reusable skill, or ordinary application code.
A Tool Is Not Necessarily an Agent
An agent generally makes decisions about what to do next. It may select tools, inspect results, revise a plan, and continue until it reaches an outcome.
A tool has a narrower contract. Given valid input, it performs a defined operation and returns a result.
For example, checking an invoice balance does not require an independent agent if the task is simply:
{ "tool": "get_invoice_balance", "input": { "invoiceId": "INV-2048" }}A normal API can handle that operation. Giving it an agent identity introduces another model decision without necessarily adding useful reasoning.
An invoice investigation is different. If the task requires reconciling inconsistent records, checking several data sources, and explaining a discrepancy, a specialist agent may have a meaningful role.
This distinction is consistent with Anthropic's engineering guidance on building effective agents , which recommends starting with simpler patterns and introducing more autonomous behavior when the task requires it.
A good default is to make deterministic operations tools and reserve agents for work that benefits from independent reasoning.
The Hidden Cost of Every Handoff
Consider a user request that takes one agent three model calls and two API calls to complete.
A multi-agent version might require a supervisor to classify the request, call a specialist, review the response, call a second specialist, and generate the final answer.
Even if each individual agent performs well, the full request now has more steps.
The extra cost comes from more than tokens. Each handoff may introduce:
- Another model invocation and its associated latency.
- Repeated context sent to a collaborator.
- Additional tool calls or network requests.
- More traces and logs to inspect during failures.
- Another decision that could route the task incorrectly.
A useful approximation is:
Total request time = routing time + specialist execution time + coordination time + final response timeParallel execution can reduce elapsed time when independent specialists work simultaneously. Sequential delegation tends to add latency unless specialization substantially reduces the work performed at each step.
For a customer-facing assistant with a strict response-time target, that difference matters.
Five Agents Can Make One Small Mistake Expensive
Suppose a supervisor sends a refund question to a payment agent. The payment agent retrieves a transaction but interprets the wrong identifier. The supervisor accepts that result and sends it to a response agent, which produces a polished explanation.
The system may return a convincing answer based on an incorrect intermediate result.
Adding a second or third agent does not automatically validate the first agent's work. In some cases, it creates more places where an error can be introduced or repeated.
This is particularly risky when agents exchange free-form text rather than structured results.
Instead of passing:
The payment looks fine, and I think the customer can probably receive a refund.
Prefer a constrained response contract:
{ "transactionId": "TXN-1042", "paymentStatus": "CAPTURED", "refundEligible": true, "refundAmount": 75.00, "currency": "USD", "evidence": [ "payment_record", "refund_policy" ]}The supervisor can validate required fields and make an explicit decision about the next step. It should not treat a collaborator's natural-language confidence as proof that the underlying operation succeeded.
Structured outputs do not eliminate model errors, but they make those errors easier to detect and test.
When Specialization Actually Helps
There are good reasons to separate agents. The strongest ones usually involve boundaries that exist outside the model itself.
Consider a financial services assistant that supports account inquiries, loan applications, and transaction investigations.
The account inquiry agent may need read-only access to customer profiles. The loan agent may need access to application documents and lending rules. A transaction investigation agent may need separate audit data and approval procedures.
Those are meaningfully different responsibilities.
An architecture might look like this:
User | v Coordinator | +-----------+-----------+ | | | v v v Accounts Lending Investigation Agent Agent Agent | | | v v v Read-only Application Audit data account API documents and controlsSeparate agents can help when teams need independent release cycles, different model configurations, isolated tool access, or distinct evaluation datasets.
AWS documents several multi-agent architecture patterns with Amazon Bedrock AgentCore Runtime , including colocated subagents and separately deployed agents. The choice affects isolation, network overhead, scaling, and operational ownership.
The number of agents should follow these requirements rather than become a target of its own.
Keep Deterministic Work Outside the Agent Conversation
An AI agent may decide that a refund request requires policy verification. That does not mean another agent should decide whether a payment was captured or whether a refund has already been issued.
Those facts should come from authoritative systems.
A safer workflow separates interpretation from execution:
User request | vAI interprets intent | vApplication validates inputs | vPolicy and authorization checks | vPayment API performs approved action | vAI explains the resultThis matters even more when several agents participate. If every collaborator can independently initiate side effects, the system needs a clear answer to questions such as who owns the transaction, who can retry it, and who records completion.
For consequential actions, the application should enforce permissions, idempotency, and approval requirements outside the model's instructions.
Agent specialization is not a substitute for conventional security and transaction controls.
Shared Context Can Become a Coordination Problem
A single agent has access to the context supplied within its session. Multiple agents may each receive a different portion of the conversation.
That creates a tradeoff.
Sharing the entire conversation with every specialist can increase token usage and expose information that a specialist does not need. Sharing too little can cause collaborators to miss an important constraint.
Imagine a customer saying:
"I want to change the delivery address, but only if the order has not shipped."
If the shipping agent receives only "change delivery address," it may miss the condition that makes the request safe.
The supervisor should pass a structured task that preserves the relevant constraint:
{ "task": "change_delivery_address", "orderId": "ORDER-2048", "newAddress": { "city": "Austin", "country": "US" }, "precondition": { "shipmentStatus": "NOT_SHIPPED" }}The receiving component should verify the precondition against the authoritative order system rather than trusting that the supervisor has already checked it.
Clear task contracts are often more valuable than giving every collaborator a large conversation history.
Watch What Happens to Streaming and Response Time
Latency problems can become particularly visible in conversational applications.
A user-facing agent may support token-by-token streaming, but the surrounding workflow can still hold back the response while earlier agents complete their work.
For example:
User | vIntent Agent 400 ms | vResearch Agent 1200 ms | vPolicy Agent 700 ms | vResponse Agent starts streamingThese durations are illustrative, not benchmark results. The point is that the user sees no generated response until the required upstream work finishes.
A practical LangGraph streaming investigation describes how nesting an agent inside a graph affected token delivery in a real-time conversational workflow. It is a useful reminder that a component's standalone streaming behavior may not survive unchanged when placed inside a larger agent system.
For interactive applications, measure time to first token alongside total completion time. A design that produces a good answer eventually may still feel unresponsive if the first visible output arrives too late.
The Supervisor Can Become the New Bottleneck
A supervisor agent is often presented as the component that makes multi-agent systems manageable.
It can also become a single coordination point through which every task must pass.
If the supervisor repeatedly examines results, chooses another collaborator, and reconstructs context, its own model calls may dominate the request.
This is especially wasteful when the routing decision is already known.
For example, an application may know from the API endpoint that a request concerns billing. It does not always need a model to rediscover that fact.
def choose_handler(request): route = request.get("route") if route == "billing": return "billing_handler" if route == "orders": return "order_handler" return "general_agent"This simplified example is not an AWS deployment recipe. It illustrates that deterministic routing can remain in application code while ambiguous requests are sent to a model for interpretation.
If the application can identify the correct destination reliably from structured input, replacing that rule with another agent call may increase cost without improving outcomes.
AgentCore Gives You Deployment Choices, Not a Reason to Split Everything
For new AWS agent applications, Amazon Bedrock AgentCore provides building blocks for hosting and operating agents across different frameworks and models.
One architectural choice is whether multiple logical agents should run together or as separate deployed services.
Colocated subagents can share a runtime process, which reduces network overhead and keeps closely related behavior within one deployment. Separate runtimes can provide stronger isolation and allow different teams or workloads to scale independently.
Neither topology is universally better.
Option A: Shared runtime
AgentCore Runtime | +---- Coordinator +---- Research helper +---- Validation helperOption B: Separate runtimes
Coordinator Runtime | +----> Research Runtime | +----> Validation RuntimeIf two helpers are always called together, use the same tools, and belong to the same team, a shared runtime may be easier to operate.
If one agent handles sensitive financial records and another serves general product documentation, separate runtime and permission boundaries may be warranted.
The important distinction is between a logical agent boundary and a deployment boundary. They do not always need to be the same thing.
Measure the Whole Request, Not Individual Agent Scores
A specialist agent can score well in isolation while the full system performs poorly.
For example, a billing agent might answer invoice questions accurately when given a clean prompt. In production, the supervisor may send it incomplete context or route the wrong request to it.
Testing the billing agent alone will not reveal those problems.
A useful evaluation plan compares complete architectures against the same request set.
Measure | Why it matters |
|---|---|
Task completion rate | Did the user get the correct outcome? |
End-to-end latency | How long did the full workflow take? |
Time to first token | How quickly did the user see a response? |
Model usage cost | How much did the complete request consume? |
Incorrect tool actions | Did the system attempt an unsafe operation? |
Routing errors | Did the supervisor choose the wrong specialist? |
Recovery rate | Could the workflow handle partial failures? |
Test at least three candidates when the problem warrants it: one agent with tools, one agent with reusable skills or helpers, and a multi-agent design.
Use the same inputs, expected outcomes, model settings where comparable, and operational constraints. Include requests that require multiple domains, ambiguous user instructions, tool failures, and attempts to trigger unauthorized actions.
The Amazon Bedrock AgentCore documentation is the starting point for checking the current service capabilities available for these architectures.
Without end-to-end measurements, it is easy to mistake a more elaborate diagram for a better system.
When I Would Choose One Agent, and When I Would Choose Five
A single agent is usually my starting point when the application has a small number of tools, one team owns the workflow, and most requests can be completed without independent specialist reasoning.
I would consider a small number of specialist agents when the application has genuinely different knowledge domains, permissions, or ownership boundaries. I would also look for evidence that specialization improves task completion enough to justify the additional latency and operational work.
Five agents might be appropriate for a complex investigation where independent specialists can analyze evidence in parallel and a coordinator combines their findings. Five agents would be harder to justify for a customer support flow that mainly retrieves records and calls a few APIs.
The decision is less about the count and more about the responsibilities.
Before adding an agent, I would ask:
- Does this component need to reason independently, or can it be a tool?
- Does it require separate permissions or data access?
- Can its work run independently or in parallel?
- Does it have a clear input and output contract?
- Can we evaluate its contribution to the complete task?
- What happens if it times out or returns an incorrect result?
- Does the additional agent improve the measured outcome?
If the answers are unclear, another agent is probably premature.
Build the Smallest Agent System That Can Do the Job
Multi-agent architecture can be useful, especially when a problem involves separate domains, security boundaries, and genuinely independent reasoning tasks.
It can also turn a manageable application into a network of model calls that is expensive to test, slow to respond, and difficult to debug.
Start with one agent and well-defined tools. Add reusable skills or helper components when the work becomes too broad. Introduce separate agents when there is a measurable benefit from specialization, isolation, or parallel execution.
Then test the entire request path, including routing, tool calls, recovery, latency, and final correctness.
The goal is not to build an AI team with the most members. It is to build a system where every member has a reason to exist.
Series: My Cloud Experience (3 articles)
- 1One AI Agent or Five? When Multi-Agent Architecture Becomes a Liability This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article