AWS Builder Center

Building multi-tenant agents on AWS

Learn how to build multi-tenant agents on AWS

Introduction

Generative AI is rapidly transforming the way we build and interact with software. At the heart of this transformation lies the concept of AI agents. These autonomous, intelligent entities are capable of perceiving their environment, making decisions, and taking actions to achieve specific goals. As organizations increasingly adopt AI-driven solutions, the need for efficient agent architectures in terms of scalability, cost, and flexibility becomes clear. Achieving these outcomes requires architecture designs that can deliver on these goals, such as multi-tenant architectures with multi-tenant agents. This blog starting from the fundamentals will try to explain concepts of multi-tenant agents.

What Are Agents?

Agents are autonomous systems designed to perform tasks on behalf of users without direct human intervention. Their core capabilities include:
  • Planning & Reasoning: Agents can break down complex tasks into subtasks, apply logic, and make decisions, often powered by large language models (LLMs).
  • Memory: Enables agents to maintain context across interactions, which is crucial for personalization, continuity, and intelligent decision-making. With short-term memory, agents can track conversation flow or task progress, while long-term memory allows them to retain user preferences, past interactions, or domain knowledge. This helps agents avoid repetition, build rapport, and deliver context-aware responses that evolve over time.
  • Tool Integration: Agents can extend their functionality to suit diverse business needs with tool integrations through which they can interact with APIs or external services.

When Should You Use Agents?

Agents are particularly valuable when:
  • Handling Complex Tasks: Tasks requiring autonomous operation, planning, and reasoning benefit from agent-driven automation. Agents are a great fit for non deterministic tasks which are probabilistic with no fixed outputs. For example, a retail company can use Agents for strategic business planning like analyzing their sales data, predicting seasonal demand, recommending optimal inventory levels and order products for their inventory for an upcoming sales event.
  • Dynamic Adaptability: Agents excel in scenarios where real-time decision-making and adaptability are required, such as the same above retail company agent in real-time can see that ordered products are going to have delays due to shipping or vendor issues, then the agent can devise an alternate plan or options.
  • Automating Repetitive Tasks: Agents can efficiently execute repetitive processes like generating reports, data retrieval, generating code, report summarization, etc. For example, an agent can assist healthcare providers by analyzing patient records, identifying potential diagnoses based on symptoms, and suggest/summarize treatment plans.
However, it's essential to weigh the cost implications of agent usage, as their advanced capabilities can introduce additional operational expenses.

Multi-tenant agents

A multi-tenant agent is an agent designed to serve multiple independent users or organizations—known as tenants—within a multi-tenant environment, while ensuring that each tenant’s data, configuration, and operations remain logically isolated. This can be achieved in two ways:
  • Pooled model: A single instance of the agent serves multiple tenants, customizing responses based on tenant-specific context, while maintaining strict data separation and isolation.
  • Silo model: Each tenant has a dedicated agent instance running in an isolated environment, with separate resources such as compute and application layers, ensuring complete operational and data isolation.
Multi-tenant agents are essential for scalable, cost-effective, and secure AI deployments, enabling centralized management and resource sharing without compromising privacy or tenant boundaries.

Amazon Bedrock agents

 Amazon Bedrock Agents  enable you to build, configure, and deploy autonomous AI agents that automate tasks and interact with users based on your organization’s data and APIs. These agents orchestrate interactions between foundation models (FMs), data sources, software applications, and user conversations. They can automatically call APIs, invoke knowledge bases, and manage complex workflows without requiring you to provision infrastructure or write extensive custom code

Silo bedrock agent


When you configure an Amazon bedrock agent, you need to configure the LLM and configure the agents functionality through action groups (which will be executed in a  AWS lambda  function). You can also enrich the agent by instructing it to interact with tenant-specific resources like  Amazon Bedrock knowledge base  or  Amazon DynamoDB , etc.
You can create a silo bedrock agent for a tenant by configuring it to only interact with tenant-specific resources. To ensure tenant resources are protected, you need to enforce tenant isolation by attaching an execution role to the agent which will only allow it to interact with the dedicated tenant-specific resources(as shown in figure 1)
In this silo model, you can attach a Amazon Bedrock knowledge base directly to an agent. Once you attach the knowledge base directly to the agent, then based on user prompt, the agent can use its reasoning ability and retrieve required relevant data from the attached knowledge base. For a silo bedrock agent you can directly attach a tenant-specific knowledge base, and leverage the agents reasoning ability to let it retrieve tenant-specific data from the knowledge base. Refer to this  gitrepo for amazon bedrock silo agent implementation details   which interacts with Amazon DynamoDB .

Pooled bedrock agent


You can also configure a pooled bedrock agent, where you configure an LLM, action groups, and the agent to interact with a pooled knowledge base. Since the pooled knowledge base has multiple tenants data, we need to have more fine grained control over what data the agent can retrieve from the pooled knowledge base based on tenant context. With a pooled agent, instead of attaching the pooled knowledge base directly to agent and leveraging the agents reasoning ability to retrieve tenant-specific data, you need to enrich the action groups to select a pooled knowledge base based on tenant context provided to it. Based on the tenant metadata provided to it, the agent can use  metadata filtering  logic (packaged in a lambda layer) to retrieve tenant-specific data from a pooled knowledge base. Here tenant context and tenant metadata can be passed to agents through session attributes.
Since the pooled agent is shared across multiple tenants, you must enforce tenant isolation at run-time by dynamically generating tenant scoped credentials which provide access only to tenant-specific resources. One way you can implement this is (as shown in figure 2) at the  Amazon API Gateway  using a AWS Lambda authorizer you can assume a attribute-based access control (ABAC)  IAM  role with the tenant context, then you can pass this temporary tenant-specific scope credentials and additional tenant metadata to agents using session attributes. You can get the tenant context and tenant metadata from your  SaaS identity  (if you have implemented it) or get just tenant context from SaaS identity then get additional tenant metadata from SaaS control plane. Refer to this  gitrepo for amazon bedrock pooled agent implementation details which interacts with Amazon DynamoDB. 

Bridge bedrock agent


You can also build a bridge bedrock agent (combining silo & pool model), where you can have a pooled agent which is shared across multiple agents which can interact with siloed tenant-specific resources. To enforce tenant isolation here, to pool agent we can pass tenant context and dynamically generated tenant-specific scope credentials which will be used by the pooled agent to interact with tenant-specific resources like Amazon knowledge base(as shown in figure 3), Amazon DynamoDB..etc
You can use these Amazon Bedrock agents to build and deploy AI agents with a fully managed experience—eliminating the need to manage infrastructure or manually integrate core agent capabilities. These agents come equipped with features like multi-agent collaboration, observability, code interpretation, memory management, guardrails, and native integration with knowledge bases.

Agents with tool use


You can also build agents using tools, agents get additional functionality by integrating with tools which can enable agents to interact with an database, an API, third party system..etc. With the advent of Model Context Protocol (MCP), it provided a standard way for agents to interact with these tools. Using  Strands Agents SDK  (which is an open-source framework developed by AWS that simplifies the building, deployment, and scaling of intelligent, autonomous AI agents capable of reasoning, planning, and interacting with external tools and APIs—powered by LLMs), you can create agents which can integrate with MCP-based tools as shown in figure 4. Here MCP server can be run on any of the AWS compute services like  Amazon EKS ,  Amazon ECS ,  AWS Lambda  and  Amazon EC2 .
You can use the Strands Agents SDK to build highly customizable agents, support integration with any model provider or proprietary LLM gateway, and the ability to deploy them anywhere you would host a Python application.

Agents with tool use - silo agent


As shown in figure 5, you can build a silo agent which can interact with a silo MCP server which has silo tools, which can interact with dedicated tenant-specific resources. Since MCP server will run on an instance of compute, you can enforce tenant isolation by attaching a execution role to the compute scoping down permissions for the tools to only interact with tenant-specific resources.

Agents with tool use - pooled agent


As shown in figure 6, you can build a pooled agent which can interact with a pooled MCP server and it’s pooled tools, which can interact with pooled resources. Here since the MCP server is shared across multiple tenants, to enforce tenant isolation you can enrich the pooled MCP server to dynamically generate tenant-specific scoped credentials in run-time by assuming an Attribute based access control (ABAC) IAM role using the tenant context passed to it by the agent. Then the pooled tools can use these tenant-specific scoped credentials to interact only with tenant-specific resources.
However, the introduction of an MCP server might add latency, and doing an assume role for each request also adds even more latency. One way to counter this is, as shown in figure 2 above, is to generate tenant-specific scoped credentials at the Amazon API Gateway level, cache them using the Amazon API Gateway caching feature and pass them to agents. And then agents can pass these attributes (which contains tenant-specific scoped credentials) to the pooled MCP server and the pooled tools can use those tenant-specific scoped credentials to enforce tenant isolation. The way you implement this might differ from architecture to architecture.

Sample implementation

Here is a sample implementation of the above outlined pooled agent which can integrate with pooled tools through a pooled MCP server using Strands Agent SDK.
Here the pooled agent creates/retrieves product details from a pooled Amazon DynamoDB table. It sends a request to the pooled MCP server, passing the tenant context. The MCP server then assumes a tenant-scoped ABAC role, which provides credentials restricted to that tenant’s data. These scoped credentials are used by the tools within the MCP server to securely access the pooled Amazon DynamoDB table with tenant-level isolation.

Attribute based access control (ABAC) role

The following ABAC role grants access exclusively to tenant-specific items in the pooled Amazon DynamoDB table, where the tenant id serves as the partition key to enforce data isolation.
plaintext
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:UpdateItem"
],
"Resource": "arn:aws:dynamodb:us-east-1:905418320997:table/pooled-tenant-table",
"Condition": {
"ForAllValues:StringEquals": {
"dynamodb:LeadingKeys": [
"${aws:PrincipalTag/tenantId}"
]
}
}
}
]
}

Pooled MCP server

Below is the implementation of the pooled MCP server, built using  FastMCP . When an agent invokes a tool, the tool uses the provided tenant id to assume the ABAC role described above. This generates tenant-scoped credentials, which are then used to interact with the pooled Amazon DynamoDB table—ensuring access is limited to items belonging to the specific tenant. This approach enforces strong tenant isolation within a shared data store. The pooled MCP server can be deployed on Amazon ECS or Amazon EKS for scalable and managed execution.
plaintext
"""
MCP Server for tenant-specific Amazon DynamoDB access using FastMCP and ABAC.
"""

import json
import os
import sys
import boto3
import uuid
import re
import logging
from datetime import datetime
from typing import Dict, List, Any, Optional, TypeVar, Generic
from pydantic import BaseModel
from fastmcp import FastMCP

# Import configuration (assumed to be defined in config.py)
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from config import MCP_SERVER_NAME, MCP_SERVER_DESCRIPTION, ROLE_ARN_TEMPLATE, DYNAMODB_TABLE_NAME

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

T = TypeVar('T')

class MCPResponse(BaseModel, Generic[T]):
result: Optional[T] = None
error: Optional[str] = None
status_code: int = 200

model_config = {"arbitrary_types_allowed": True}

def dict(self) -> Dict[str, Any]:
response = {}
if self.result is not None:
response["result"] = self.result
if self.error is not None:
response["error"] = self.error
return response

class TenantItem(BaseModel):
tenant_id: str
item_id: str
product: str
value: Any
description: Optional[str] = None
created_at: str

class TenantCredentialManager:
def __init__(self, role_arn_template: str):
self.role_arn_template = role_arn_template
self.sts_client = boto3.client('sts')

def get_tenant_credentials(self, tenant_id: str) -> Dict[str, str]:
if not re.match(r'^[a-zA-Z0-9-]{1,36}$', tenant_id):
raise ValueError(f"Invalid tenant ID format: {tenant_id}")
try:
response = self.sts_client.assume_role(
RoleArn=self.role_arn_template,
RoleSessionName=f"tenant-{tenant_id}-session",
DurationSeconds=3600,
Tags=[{'Key': 'tenantId', 'Value': tenant_id}]
)
credentials = response['Credentials']
return {
'aws_access_key_id': credentials['AccessKeyId'],
'aws_secret_access_key': credentials['SecretAccessKey'],
'aws_session_token': credentials['SessionToken']
}
except Exception as e:
raise Exception(f"Failed to assume role for tenant {tenant_id}: {str(e)}")

class TenantDynamoDBManager:
def __init__(self, table_name: str):
self.table_name = table_name

def get_dynamodb_client(self, credentials: Dict[str, str]):
return boto3.client(
'dynamodb',
aws_access_key_id=credentials['aws_access_key_id'],
aws_secret_access_key=credentials['aws_secret_access_key'],
aws_session_token=credentials['aws_session_token']
)

def get_tenant_items(self, dynamodb_client, tenant_id: str) -> List[Dict]:
try:
response = dynamodb_client.query(
TableName=self.table_name,
KeyConditionExpression="tenant_id = :tid",
ExpressionAttributeValues={":tid": {"S": tenant_id}}
)
items = []
for item in response.get('Items', []):
parsed_item = {k: v[list(v.keys())[0]] for k, v in item.items()}
items.append(parsed_item)
return items
except Exception as e:
logger.error(f"Failed to query items for tenant {tenant_id}: {str(e)}")
raise

def put_tenant_item(self, dynamodb_client, tenant_id: str, item_data: Dict) -> Dict:
try:
item_with_tenant = item_data.copy()
item_with_tenant['tenant_id'] = tenant_id
item_with_tenant['item_id'] = item_with_tenant.get('item_id', str(uuid.uuid4()))
item_with_tenant['created_at'] = datetime.now().isoformat()
dynamodb_item = {
k: {"S": str(v)} if isinstance(v, (str, dict, list)) else
{"N": str(v)} if isinstance(v, (int, float)) else
{"BOOL": v} if isinstance(v, bool) else
{"S": str(v)}
for k, v in item_with_tenant.items()
}
dynamodb_client.put_item(TableName=self.table_name, Item=dynamodb_item)
return item_with_tenant
except Exception as e:
logger.error(f"Failed to put item for tenant {tenant_id}: {str(e)}")
raise

app = FastMCP(
title=MCP_SERVER_NAME,
description=MCP_SERVER_DESCRIPTION
)

credential_manager = TenantCredentialManager(ROLE_ARN_TEMPLATE)
dynamodb_manager = TenantDynamoDBManager(DYNAMODB_TABLE_NAME)

@app.tool(
name="get_tenant_items",
description="Get items for a specific tenant from the pooled DynamoDB table"
)
def get_tenant_items(tenant_id: str) -> MCPResponse[List[TenantItem]]:
try:
credentials = credential_manager.get_tenant_credentials(tenant_id)
dynamodb_client = dynamodb_manager.get_dynamodb_client(credentials)
items = dynamodb_manager.get_tenant_items(dynamodb_client, tenant_id)
tenant_items = [
TenantItem(
tenant_id=item.get("tenant_id"),
item_id=item.get("item_id"),
product=item.get("product"),
value=item.get("value"),
description=item.get("description"),
created_at=item.get("created_at")
) for item in items
]
return MCPResponse(result=tenant_items)
except ValueError as e:
return MCPResponse(error=str(e), status_code=400)
except Exception as e:
return MCPResponse(error=str(e), status_code=500)

@app.tool(
name="put_tenant_item",
description="Add an item for a specific tenant to the pooled DynamoDB table"
)
def put_tenant_item(tenant_id: str, product: str, value: int) -> MCPResponse[TenantItem]:
try:
credentials = credential_manager.get_tenant_credentials(tenant_id)
dynamodb_client = dynamodb_manager.get_dynamodb_client(credentials)
item_data_dict = {"product": product, "value": value}
created_item = dynamodb_manager.put_tenant_item(dynamodb_client, tenant_id, item_data_dict)
tenant_item = TenantItem(**created_item)
return MCPResponse(result=tenant_item)
except ValueError as e:
return MCPResponse(error=str(e), status_code=400)
except Exception as e:
return MCPResponse(error=str(e), status_code=500)

if __name__ == "__main__":
app.run('streamable-http', host="0.0.0.0", port=8000, path='/mcp')

Pooled agent

Below is the code for the pooled agent, developed using the Strands Agents SDK. It initializes an MCP client to connect to the above described pooled MCP server, and then configures the agent with the desired model and a set of tools available from the pooled MCP server.
plaintext
import os
import json
import re
import uuid
import logging
from typing import Dict, Any, List, Optional
from pydantic import BaseModel, Field
from mcp.client.streamable_http import streamablehttp_client
from strands.agent.conversation_manager import ConversationManager, SlidingWindowConversationManager

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Import configuration
from config import MCP_SERVER_URL, AGENT_NAME, AGENT_DESCRIPTION, BEDROCK_REGION, BEDROCK_MODEL_ID

# Import Strands Agents SDK
from strands import Agent
from strands.tools.mcp import MCPClient

async def main(input_text: str = None):
"""
Main function to create and run the agent.
Args:
input_text: Optional input text to process. If None, runs in interactive mode.
"""
logger.info(f"Starting Bedrock Agent for MCP Server at {MCP_SERVER_URL}")
# Create agent
mcp_client = MCPClient(lambda: streamablehttp_client(MCP_SERVER_URL))

with mcp_client:
# Create agent
agent = Agent(
model=BEDROCK_MODEL_ID,
system_prompt="You help users to create products and get product details.
",
tools=mcp_client.list_tools_sync(),
conversation_manager=CustomConversationManager()
)
tenant_id = uuid.uuid4()
input_text_with_context = f'{input_text} for tenant id {tenant_id}'
response = agent(input_text_with_context)
logger.info(response)

# Entry point
if __name__ == "__main__":
try:
import asyncio
asyncio.run(main())
except Exception as e:
logger.error(f"Failed to run agent: {str(e)}")
import traceback
logger.error(traceback.format_exc())

class CustomConversationManager(SlidingWindowConversationManager):

def __scrub_tenant_id(self, messages, uuid_pattern=r'[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}'):
def scrub_value(value):
if isinstance(value, str):
return re.sub(uuid_pattern, '<tenant_id_scrubbed>', value)
elif isinstance(value, list):
return [scrub_value(v) for v in value]
elif isinstance(value, dict):
return {k: scrub_value(v) for k, v in value.items()}
else:
return value

for i, msg in enumerate(messages):
messages[i] = scrub_value(msg)
def apply_management(self, messages):
self.__scrub_tenant_id(messages=messages)
return super().apply_management(messages)
In this above example, the tenant context (tenant id) is passed as part of the input prompt. Since the pooled agent is shared across multiple tenants, it's important to ensure that the tenant context does not persist in the agent's conversation history. To address this, the Strands Agents SDK offers a flexible framework for context management via the   ConversationManager   abstract class, which supports various strategies for managing conversation history.
One such example is the   SlidingWindowConversationManager  , which can be extended to customize behavior. In the code above, a  CustomConversationManager  extends   SlidingWindowConversationManager   and implements logic to scrub tenant context from the conversation history. This custom manager is then configured when initializing the agent, ensuring secure and isolated interactions across tenants.

Conclusion

Whether you're looking for a fully managed solution or a highly customizable framework, on AWS you have flexible options for building multi-tenant agent-based systems. Amazon Bedrock agents offer a streamlined, infrastructure-free path with built-in capabilities like multi-agent collaboration, memory, and guardrails, while the Strands Agents SDK provides deep customization, model-provider flexibility, and deployment freedom for more advanced or specialized use cases.
As outlined above, you can develop multi-tenant agents with Amazon Bedrock Agents and/or the Strands Agents SDK. Then with these multi-tenant agents and agentic frameworks you can start building multi-tenant multi-agent architectures.  
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article