AWS Builder Center
AgentGarage

AgentGarage

AgentGarage is a behavioural-coverage platform for AI agents. It learns from private production traces, maps agent decisions, finds unexplored and high-risk paths, and generates realistic test scenarios. These scenarios run against the real agent in an isolated AWS sandbox with mocked tools. Confirmed failures become permanent regression tests. Deployed with AWS CDK, it supports BYOK models, secure data ownership, and continuous testing across prompts, models, tools, and code.

Student
My Contributions to AgentGarage CDK
AgentGarage is designed to make AI agents safer, more observable, and easier to test. My contribution focused on redesigning its AWS CDK infrastructure and improving the complete agent-testing workflow.
The architecture now uses one main stack with eight nested stacks: Network, Storage, Graph, Intake, Compute, Orchestration, API, and Frontend. OpenSearch was replaced with Neptune Serverless to represent agent behaviour as a property graph. States are stored as vertices, while actions are represented as edges containing agent IDs, status, and execution counts.
I redesigned the model execution path by removing the GPU-based EC2 world-model host. The new simulation Lambda uses Amazon Bedrock by default, keeping private model execution inside AWS. It also supports multi-tenant BYOK, allowing each tenant to connect a personal model provider through Secrets Manager. Sensitive keys are never stored directly in DynamoDB.
The trace-ingestion pipeline now uses an ECS collector, Kinesis, a redaction Lambda, S3, and DynamoDB. KMS encryption, dead-letter queues, batching, and private VPC endpoints improve security and reliability.
The sandbox workflow uses Step Functions and Fargate to generate, run, evaluate, and persist agent scenarios. Failed invariants become regression tests.
Overall, my work helps AgentGarage discover behavioural gaps, reproduce failures, and measure coverage improvements across AI-agent workflows.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article