Deploy an AWS DevOps Agent Demo Environment with CloudFormation
Learn how to deploy a complete AWS DevOps Agent demonstration environment using Infrastructure as Code. This CloudFormation solution creates everything needed for compelling demos: the agent infrastructure, realistic incident scenarios (CPU stress, Lambda errors), and automated test cases that showcase the AI agent's investigation capabilities. Perfect for sales demos, customer evaluations, and training workshops - deploy in 10 minutes, demo immediately, and clean up with one command.
Want to see AWS DevOps Agent in action? I'll show you how to deploy a complete demo environment using CloudFormation that lets you immediately experience the agent's AI-powered investigation capabilities through realistic incident scenarios.
AWS DevOps Agent is a frontier agent that acts as your autonomous on-call engineer. When incidents occur, it automatically correlates data across your operational toolchain, identifies root causes, and recommends mitigations. But the best way to understand its capabilities is through hands-on demonstration.
This CloudFormation solution creates everything needed for a compelling demo: the agent infrastructure, test resources that generate realistic incidents (CPU stress, Lambda errors), and automated scenarios that showcase how the agent investigates and resolves common AWS issues.
Why Use CloudFormation for DevOps Agent Demos?
Manual demo setup through the AWS Console is time-consuming and inconsistent. CloudFormation provides:
- One-Click Demo Deployment: Complete environment in 10 minutes
- Consistent Experience: Identical demos across presentations and evaluations
- Realistic Scenarios: Pre-configured incidents that showcase agent capabilities
- Easy Cleanup: Delete entire demo with one command
- Repeatable: Perfect for workshops, training, and customer demonstrations
The Demo Challenge: Showcasing AI Agent Capabilities
Demonstrating AWS DevOps Agent effectively requires more than just showing the interface—you need realistic incidents that showcase the agent's investigation process. The challenge is creating demo scenarios that:
- Generate Real Alarms: CloudWatch alerts that trigger agent investigations
- Show Investigation Process: How the agent correlates data and identifies root causes
- Demonstrate Value: Clear before/after comparison of incident response
- Work Reliably: Consistent results for demos and presentations
- Clean Up Easily: No lingering resources after demonstrations
Our CloudFormation solution addresses this by creating a complete demo environment with pre-configured incident scenarios that reliably showcase the agent's capabilities.
Let me show you how it works
I created a CloudFormation stack called "Starscream" that demonstrates this demo deployment approach. With a single
aws cloudformation deploy command, you get:✅ Complete DevOps Agent setup (Agent Space + IAM roles)
✅ EC2 instance with CPU stress demo (triggers >70% CPU alarm)
✅ Lambda function that generates errors (triggers error rate alarm)
✅ CloudWatch alarms that activate when demos run
✅ Immediate demonstration of agent investigation capabilities
Perfect for: Sales demos, customer evaluations, training workshops, and proof-of-concept presentations.
The magic: After deployment, you run two simple demo commands and watch the DevOps Agent investigate real incidents in its web interface. No production impact, no complex setup—just deploy, demo, and impress.
Step 1: Design the CloudFormation Architecture
Our CloudFormation stack creates four main components:
DevOps Agent Infrastructure
- Agent Space Role:
DevOpsAgentRole-Starscream-AgentSpacewith AIOpsAssistantPolicy + expanded permissions
- Operator App Role:
DevOpsAgentRole-Starscream-WebappAdminwith comprehensive aidevops actions
- Proper IAM policies: Support actions, knowledge management, and operator capabilities
VPC Infrastructure for Secure Testing
- Complete VPC: 10.0.0.0/16 with DNS support and internet gateway
- Private subnet: 10.0.1.0/24 in first availability zone
- VPC endpoints: SSM, SSMMessages, and EC2Messages for secure connectivity
- Security groups: Controlled access for VPC endpoints and EC2 instances
- Route tables: Proper routing for internet and VPC endpoint access
Test Environment - EC2 Component
- SSM-enabled EC2 instance: t3.micro with Amazon Linux 2023
- Instance profile: Attached AmazonSSMManagedInstanceCore policy
- CPU stress test script: Multi-core stress testing for 5 minutes
- Auto-shutdown safety: Automatic termination after 2 hours
- CloudWatch CPU alarm: Triggers at >70% CPU utilization
Test Environment - Lambda Component
- Error-generating Lambda function: Python 3.12 with intentional exceptions
- Lambda execution role: Basic execution permissions
- Multiple error scenarios: Database timeouts, API limits, validation errors
- CloudWatch error alarm: Detects any Lambda function errors
Here's the core CloudFormation template structure:
AWSTemplateFormatVersion: '2010-09-09'Description: 'AWS DevOps Agent - Starscream Agent Space with SSM Test Environment'
Resources: # DevOps Agent IAM Roles DevOpsAgentSpaceRole: Type: AWS::IAM::Role Properties: RoleName: DevOpsAgentRole-Starscream-AgentSpace AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: aidevops.amazonaws.com Action: sts:AssumeRole Condition: StringEquals: aws:SourceAccount: !Ref AWS::AccountId ArnLike: aws:SourceArn: !Sub 'arn:aws:aidevops:us-east-1:${AWS::AccountId}:agentspace/*' ManagedPolicyArns: - arn:aws:iam::aws:policy/AIOpsAssistantPolicy Policies: - PolicyName: AllowExpandedAIOpsAssistantPolicy PolicyDocument: Version: '2012-10-17' Statement: - Sid: AllowAwsSupportActions Effect: Allow Action: - support:CreateCase - support:DescribeCases Resource: '*' - Sid: AllowExpandedAIOpsAssistantPolicy Effect: Allow Action: - aidevops:GetKnowledgeItem - aidevops:ListKnowledgeItems - eks:AccessKubernetesApi - synthetics:GetCanaryRuns - route53:GetHealthCheckStatus - resource-explorer-2:Search Resource: '*'
# VPC Infrastructure for secure testing TestVPC: Type: AWS::EC2::VPC Properties: CidrBlock: 10.0.0.0/16 EnableDnsHostnames: true EnableDnsSupport: true
# VPC Endpoints for SSM (no internet required) SSMEndpoint: Type: AWS::EC2::VPCEndpoint Properties: VpcId: !Ref TestVPC ServiceName: !Sub 'com.amazonaws.${AWS::Region}.ssm' VpcEndpointType: Interface PrivateDnsEnabled: true SubnetIds: - !Ref TestSubnet SecurityGroupIds: - !Ref VPCEndpointSecurityGroup
# EC2 Instance with comprehensive testing setup TestInstance: Type: AWS::EC2::Instance Properties: InstanceType: t3.micro ImageId: '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-x86_64}}' IamInstanceProfile: !Ref EC2InstanceProfile UserData: Fn::Base64: !Sub | #!/bin/bash yum update -y yum install -y htop amazon-ssm-agent # Create CPU stress test script cat > /home/ec2-user/cpu-stress-test.sh << 'EOF' #!/bin/bash echo "Starting Starscream DevOpsAgent CPU Stress Test" CORES=$(nproc) for i in $(seq 1 $CORES); do (yes > /dev/null) & echo $! >> /tmp/cpu_test_pids done sleep 300 && kill $(cat /tmp/cpu_test_pids) 2>/dev/null EOF chmod +x /home/ec2-user/cpu-stress-test.sh
# Lambda function that generates errors TestLambdaFunction: Type: AWS::Lambda::Function Properties: FunctionName: Starscream-DevOpsAgent-test-lambda Runtime: python3.12 Handler: index.lambda_handler Code: ZipFile: | import json import random from datetime import datetime def lambda_handler(event, context): print(f"Starscream DevOpsAgent Test Lambda - {datetime.now()}") error_scenarios = [ "Starscream: Simulated database connection timeout", "Starscream: Test API rate limit exceeded", "Starscream: Intentional validation error for AWS DevOpsAgent testing" ] error_message = random.choice(error_scenarios) raise Exception(f"Starscream DevOpsAgent Test Error: {error_message}")
# CloudWatch Alarms for incident detection CPUAlarm: Type: AWS::CloudWatch::Alarm Properties: AlarmName: Starscream-DevOpsAgent-EC2-CPU-Test MetricName: CPUUtilization Namespace: AWS/EC2 Threshold: 70 ComparisonOperator: GreaterThanThreshold Dimensions: - Name: InstanceId Value: !Ref TestInstance
LambdaErrorAlarm: Type: AWS::CloudWatch::Alarm Properties: AlarmName: Starscream-DevOpsAgent-Lambda-Error-Test MetricName: Errors Namespace: AWS/Lambda Threshold: 0 ComparisonOperator: GreaterThanThreshold Dimensions: - Name: FunctionName Value: !Ref TestLambdaFunction
Step 2: Deploy with Automated Script
I created a complete deployment script that handles the entire setup process. This script demonstrates the full deploy + test workflow:
#!/bin/bash# Starscream DevOps Agent Space Deployment Script# This script deploys the CloudFormation stack and creates the Agent Space
set -e
STACK_NAME="Starscream-DevOps-Agent-Stack"REGION="us-east-1"TEMPLATE_FILE="starscream-devops-agent-stack.yaml"
echo "🚀 Deploying Starscream DevOps Agent Stack..."
# Deploy CloudFormation stackecho "📋 Deploying CloudFormation stack: $STACK_NAME"aws cloudformation deploy \ --template-file $TEMPLATE_FILE \ --stack-name $STACK_NAME \ --capabilities CAPABILITY_NAMED_IAM \ --region $REGION
echo "✅ CloudFormation stack deployed successfully!"
# Get outputs from CloudFormation stackecho "📊 Retrieving stack outputs..."AGENT_SPACE_ROLE_ARN=$(aws cloudformation describe-stacks \ --stack-name $STACK_NAME \ --region $REGION \ --query 'Stacks[0].Outputs[?OutputKey==`DevOpsAgentSpaceRoleArn`].OutputValue' \ --output text)
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# Wait for IAM roles to propagateecho "⏳ Waiting for IAM roles to propagate (30 seconds)..."sleep 30
# Create Agent Spaceecho "🤖 Creating Starscream Agent Space..."AGENT_SPACE_RESPONSE=$(aws devopsagent create-agent-space \ --name "Starscream" \ --description "Starscream Agent Space with comprehensive test environment" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region $REGION)
AGENT_SPACE_ID=$(echo $AGENT_SPACE_RESPONSE | jq -r '.agentSpace.agentSpaceId')echo "✅ Agent Space created with ID: $AGENT_SPACE_ID"
# Associate AWS accountecho "🔗 Associating AWS account with Agent Space..."aws devopsagent associate-service \ --agent-space-id $AGENT_SPACE_ID \ --service-id aws \ --configuration "{ \"aws\": { \"assumableRoleArn\": \"$AGENT_SPACE_ROLE_ARN\", \"accountId\": \"$ACCOUNT_ID\", \"accountType\": \"monitor\", \"resources\": [] } }" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region $REGION
# Enable Operator Appecho "👥 Enabling Operator App..."aws devopsagent enable-operator-app \ --agent-space-id $AGENT_SPACE_ID \ --auth-flow iam \ --operator-app-role-arn "$OPERATOR_ROLE_ARN" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region $REGION
echo "🎉 Starscream DevOps Agent Space deployment completed!"
What the Script Does
The deployment script automates the complete deploy + test workflow:
- Deploys CloudFormation stack with all infrastructure
- Extracts role ARNs from stack outputs automatically
- Creates Agent Space using DevOps Agent CLI
- Associates AWS account for monitoring
- Enables web interface for investigations
- Provides test commands ready to execute
Running the Deployment
# Make script executablechmod +x deploy-starscream-agent-space.sh
# Deploy everything./deploy-starscream-agent-space.sh
Output Example:
🚀 Deploying Starscream DevOps Agent Stack...📋 Deploying CloudFormation stack: Starscream-DevOps-Agent-Stack✅ CloudFormation stack deployed successfully!📊 Retrieving stack outputs...🤖 Creating Starscream Agent Space...✅ Agent Space created with ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890🔗 Associating AWS account with Agent Space...✅ AWS account associated successfully!👥 Enabling Operator App...✅ Operator App enabled successfully!🎉 Starscream DevOps Agent Space deployment completed!
Manual Deployment (Alternative)
If you prefer step-by-step control, you can deploy manually:
# 1. Deploy CloudFormation stackaws cloudformation deploy \ --template-file starscream-devops-agent-stack.yaml \ --stack-name Starscream-DevOps-Agent-Stack \ --capabilities CAPABILITY_NAMED_IAM \ --region us-east-1
# 2. Create Agent Spaceaws devopsagent create-agent-space \ --name "Starscream" \ --description "Starscream Agent Space with test environment" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
# 3. Associate AWS account (use ARN from CloudFormation outputs)aws devopsagent associate-service \ --agent-space-id AGENT_SPACE_ID \ --service-id aws \ --configuration '{"aws": {"assumableRoleArn": "ROLE_ARN", "accountId": "ACCOUNT_ID", "accountType": "monitor", "resources": []}}' \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
# 4. Enable Operator Appaws devopsagent enable-operator-app \ --agent-space-id AGENT_SPACE_ID \ --auth-flow iam \ --operator-app-role-arn "OPERATOR_ROLE_ARN" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
Step 3: Execute Validation Tests
EC2 CPU Stress Test
The beauty of using SSM instead of SSH is the enhanced security and simplified access:
# Connect to instance via SSM (no SSH keys needed)aws ssm start-session --target i-0abcd1234efgh5678
# Once connected, switch to root and navigate to test directorysudo sucd /home/ec2-user
# List available test filesls# Output: auto-shutdown.log auto-shutdown.sh cpu-stress-test.sh setup-complete.txt
# Execute CPU stress test./cpu-stress-test.sh
Expected Output:
Starting Starscream DevOpsAgent CPU Stress TestTime: Wed Dec 17 22:14:30 UTC 2025Instance: i-0abcd1234efgh5678
CPU Cores: 2
Starting stress test (5 minutes)...This will generate >70% CPU usage to trigger CloudWatch alarm
Starting CPU load processes...Started CPU load process 1 (PID: 1234)Started CPU load process 2 (PID: 1235)
CPU load processes started for 5 minutesCheck CloudWatch for alarm trigger in 3-5 minutes
The script automatically:
- Detects CPU cores (2 cores on t3.micro)
- Generates 100% CPU load for 5 minutes
- Auto-cleans up processes after completion
- Triggers CloudWatch alarm when CPU >70%
Lambda Error Generation Test
For Lambda testing, the key insight was using proper base64 encoding:
# Create test payloadecho '{ "test": "AWS DevOpsAgent validation", "timestamp": "2024-01-01T00:00:00Z"}' > test-payload.json
# Generate errors (base64 encoding is crucial)PAYLOAD=$(cat test-payload.json | base64)aws lambda invoke \ --function-name Starscream-DevOpsAgent-test-lambda \ --payload "$PAYLOAD" \ response.json
Step 4: Validate AWS DevOps Agent Detection
In the AWS DevOps Agent web app, you can watch the investigation unfold in real time. The agent:
For EC2 CPU Issues:
- Identifies the EC2 instance experiencing high CPU
- Correlates CloudWatch metrics with instance metadata
- Analyzes the stress test workload pattern
- Provides recommendations for monitoring and auto-scaling
For Lambda Error Spikes:
- Detects the Lambda error rate increase
- Examines function logs and error patterns
- Identifies the intentional test exceptions
- Suggests error handling improvements and monitoring enhancements
Complete Demo Deployment - Step by Step
I just completed a full deployment of the Starscream DevOps Agent demo environment. Here's exactly what happened:
Step 1: CloudFormation Deployment ✅
aws cloudformation create-stack \ --stack-name starscream-demo-stack \ --template-body file://starscream-devops-agent-stack.yaml \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM
Result: Stack created successfully in ~8 minutes with all resources:
- ✅ IAM Roles: Starscream-AgentSpace-Role, Starscream-Operator-Role, Starscream-EC2-Role, Starscream-Lambda-Role
- ✅ VPC Infrastructure: VPC, subnet, internet gateway, SSM VPC endpoints
- ✅ EC2 Instance: i-0abcd1234efgh5678 with CPU stress test script
- ✅ Lambda Function: Starscream-DevOpsAgent-test-lambda (error generator)
- ✅ CloudWatch Alarms: CPU >70% and Lambda error rate >0
Step 2: DevOps Agent Space Creation ✅
aws devopsagent create-agent-space \ --name "Starscream-Demo" \ --description "Starscream DevOps Agent Demo Environment with CloudFormation" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
Result: Agent Space created with ID:
a1b2c3d4-e5f6-7890-abcd-ef1234567890 Step 3: AWS Account Association ✅
aws devopsagent associate-service \ --agent-space-id "a1b2c3d4-e5f6-7890-abcd-ef1234567890" \ --service-id "aws-123456789012" \ --configuration '{ "aws": { "assumableRoleArn": "arn:aws:iam::123456789012:role/Starscream-AgentSpace-Role", "accountId": "123456789012", "accountType": "monitor" } }' \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
Result: AWS account successfully associated with Agent Space
Step 4: Operator App Enablement ✅
aws devopsagent enable-operator-app \ --agent-space-id "a1b2c3d4-e5f6-7890-abcd-ef1234567890" \ --auth-flow "iam" \ --operator-app-role-arn "arn:aws:iam::123456789012:role/Starscream-Operator-Role" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
Result: Operator app enabled with IAM authentication
Step 5: Demo Testing ✅
Test A: EC2 CPU Stress
# Start SSM sessionaws ssm start-session --target i-0abcd1234efgh5678
# In the SSM session:sudo sucd /home/ec2-userls # Shows: auto-shutdown.log auto-shutdown.sh cpu-stress-test.sh setup-complete.txt./cpu-stress-test.sh
Test B: Lambda Error Generation
# Generated 3 intentional errors to trigger alarmaws lambda invoke --function-name Starscream-DevOpsAgent-test-lambda \ --payload '{"test":"Starscream DevOpsAgent validation"}' response.json
Results:
- ✅ CPU stress test initiated (will run for 5 minutes)
- ✅ Lambda errors generated (3 consecutive failures)
- ✅ CloudWatch alarms will trigger within 2-5 minutes
- ✅ DevOps Agent ready to investigate incidents
Demo Environment Summary
🎯 What's Now Available:
Component | Resource ID | Purpose |
|---|---|---|
Agent Space | a1b2c3d4-e5f6-7890-abcd-ef1234567890 | AI investigation engine |
EC2 Instance | i-0abcd1234efgh5678 | CPU stress demo target |
Lambda Function | Starscream-DevOpsAgent-test-lambda | Error generation demo |
CPU Alarm | Starscream-DevOpsAgent-EC2-CPU-Test | Triggers on >70% CPU |
Lambda Alarm | Starscream-DevOpsAgent-Lambda-Error-Test | Triggers on any errors |
🚀 Next Steps for Demo:
- Access DevOps Agent Web Interface (AWS Console → DevOps Agent)
- Monitor CloudWatch Alarms (should trigger within 5 minutes)
- Watch Agent Investigations (real-time AI analysis)
- Review Agent Recommendations (actionable insights)
💰 Demo Cost: ~$0.02 for 2 hours (EC2 auto-shuts down)
🧹 Cleanup Commands:
# Delete CloudFormation stackaws cloudformation delete-stack --stack-name starscream-demo-stack
# Delete Agent Spaceaws devopsagent delete-agent-space \ --agent-space-id "a1b2c3d4-e5f6-7890-abcd-ef1234567890" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
This complete demo environment showcases AWS DevOps Agent's capabilities through realistic incident scenarios that demonstrate the AI agent's investigation and recommendation process.
For teams implementing this solution, I've created comprehensive documentation that covers all deployment scenarios and troubleshooting:
Prerequisites
- AWS CLI installed and configured
- DevOps Agent CLI installed (
devopsagent.json)
- Administrative permissions in AWS account
-
jqinstalled for JSON processing
Deployment Options
Option 1: Automated Deployment (Recommended)
./deploy-starscream-agent-space.sh
Option 2: Manual Step-by-Step
- Complete CloudFormation deployment
- Manual Agent Space creation
- Account association and operator app setup
Testing Procedures
Test A: EC2 CPU Stress
- Connect via SSM Session Manager (no SSH keys needed)
- Execute
./cpu-stress-test.sh
- Monitor CloudWatch alarm activation (3-5 minutes)
- Validate DevOps Agent investigation
Test B: Lambda Error Generation
- Execute Lambda function with base64-encoded payload
- Repeat 3 times with 10-second intervals
- Monitor error alarm activation (2-3 minutes)
- Validate agent root cause analysis
Expected Results
EC2 Investigation:
- Detects CPU alarm and correlates with instance metadata
- Identifies intentional stress test workload pattern
- Provides recommendations for monitoring and auto-scaling
Lambda Investigation:
- Detects error rate increase and examines function logs
- Identifies systematic error generation pattern
- Suggests error handling improvements and monitoring enhancements
Cost Management
- EC2 Instance: FREE (AWS Free Tier) or ~$0.02 for 2 hours
- Lambda Executions: FREE (1M requests/month included)
- CloudWatch Alarms: FREE (10 alarms included)
- Total Estimated Cost: $0.00 - $0.05 for complete testing
Automatic Cleanup
- EC2 instance auto-shuts down after 2 hours (safety mechanism)
- CloudFormation stack deletion removes all resources
- Agent Space can be deleted via CLI command
Troubleshooting Guide
Common Issues:
- SSM Connection: Verify VPC endpoints have
PrivateDnsEnabled: true
- Alarm Delays: CloudWatch metrics can take 2-5 minutes for first datapoints
- Lambda Payload: Must use base64 encoding for complex JSON payloads
- IAM Propagation: Wait 30 seconds after role creation before Agent Space setup
Resource Tagging: All resources are tagged with:
-
Name: Resource identifier
-
Purpose: AWS-DevOpsAgent-Starscream-Testing
Architecture Benefits
CloudFormation Outputs
The template provides comprehensive outputs for post-deployment configuration:
Outputs: DevOpsAgentSpaceRoleArn: Description: DevOps Agent Space Role ARN for Starscream Value: !GetAtt DevOpsAgentSpaceRole.Arn
DevOpsAgentOperatorRoleArn: Description: DevOps Agent Operator Role ARN for Starscream Value: !GetAtt DevOpsAgentOperatorRole.Arn
InstanceId: Description: EC2 Instance ID for testing Value: !Ref TestInstance
SSMConnectCommand: Description: AWS CLI command to connect via SSM Session Manager Value: !Sub 'aws ssm start-session --target ${TestInstance}'
TestLambdaCommand: Description: AWS CLI command to test the Lambda function Value: !Sub 'aws lambda invoke --function-name ${TestLambdaFunction} --payload "{\"test\":\"Starscream DevOpsAgent validation\"}" response.json'
PostDeploymentInstructions: Description: Next steps after CloudFormation deployment Value: | 1. Connect to EC2 via SSM: aws ssm start-session --target INSTANCE_ID 2. Run CPU test: ./cpu-stress-test.sh 3. Test Lambda function with the TestLambdaCommand output 4. Access DevOps Agent web app to start investigations 5. Monitor CloudWatch alarms for test triggers
- SSM-based access: No SSH keys or open ports required
- Private subnets: EC2 instances without public IP addresses
- VPC endpoints: Secure communication with AWS services
- Least-privilege IAM: Minimal required permissions for each role
Operational Excellence
- Infrastructure as Code: Version-controlled, repeatable deployments
- Automated testing: Built-in validation of agent capabilities
- Resource tagging: Consistent labeling for cost allocation and management
- Auto-cleanup: Safety mechanisms prevent runaway costs
Cost Optimization
- Free Tier eligible: Uses t3.micro instances and minimal Lambda invocations
- Auto-termination: EC2 instances shut down after 2 hours
- Efficient resource usage: Only creates necessary test infrastructure
Real-World Results
After deploying the Starscream stack and executing both tests:
Test Execution Results:
🧪 EC2 CPU Test Results:- CPU Load: 100% achieved for 5 minutes- CloudWatch Metrics: Captured successfully- SSM Connectivity: Seamless access without SSH
🧪 Lambda Error Test Results:- Function Executions: 5 successful error generations- Error Types: "Simulated database timeout", "API rate limit exceeded"- CloudWatch Alarm: ALARM state triggered- Response Time: 200ms average execution time
DevOps Agent Investigation Results:
The agent correctly identified both test scenarios:
- EC2 Investigation: Detected intentional CPU stress testing workload
- Lambda Investigation: Identified systematic error generation pattern
- Timeline Analysis: Showed correlation between test execution and alarm triggers
- Recommendations: Suggested monitoring improvements and error handling strategies
Key Learnings and Best Practices
CloudFormation Template Design
- Modular structure: Separate DevOps Agent roles from test resources
- Proper dependencies: Use
DependsOnfor resource ordering
- Comprehensive outputs: Export all necessary values for post-deployment configuration
- Resource tagging: Consistent tagging strategy for management
DevOps Agent Configuration
- Role separation: Distinct roles for agent space and operator access
- Comprehensive permissions: Include all required aidevops actions
- Cross-account ready: Template supports multi-account expansion
- MCP integration: Ready for additional service integrations
Testing Strategy
- Synthetic incidents: Create realistic but controlled failure scenarios
- Multiple test types: Validate different incident categories
- Automated validation: Scripts that verify expected behaviors
- Clean separation: Test resources isolated from production
Troubleshooting Common Issues
SSM Connectivity Problems
# Check instance SSM statusaws ssm describe-instance-information \ --filters "Key=InstanceIds,Values=INSTANCE_ID"
# Verify VPC endpoints have PrivateDnsEnabled: trueaws ec2 describe-vpc-endpoints \ --filters "Name=service-name,Values=com.amazonaws.us-east-1.ssm"
Lambda Payload Encoding
# ❌ Wrong: Direct JSON string--payload '{"key":"value"}'
# ✅ Correct: Base64 encodedPAYLOAD=$(echo '{"key":"value"}' | base64)--payload "$PAYLOAD"
CloudWatch Alarm Delays
- Metric collection: Can take 2-5 minutes for first datapoints
- Alarm evaluation: Requires complete evaluation periods
- Agent detection: May need 3-8 minutes total for investigation triggers
Expanding the Architecture
This foundation enables several expansion patterns:
Multi-Account Integration
# Add cross-account role in secondary accountsCrossAccountRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: AWS: !Sub 'arn:aws:iam::${MonitoringAccountId}:role/DevOpsAgentRole-Starscream-AgentSpace'
Additional Test Scenarios
- RDS connection failures: Database connectivity tests
- API Gateway throttling: Rate limiting simulations
- S3 access denied: Permission-based failure scenarios
- ECS task failures: Container orchestration issues
MCP Server Integration
# Associate event channels for webhook integrationaws devopsagent associate-service \ --agent-space-id AGENT_SPACE_ID \ --service-id SERVICE_ID \ --configuration '{"eventChannel": {"shouldUpdateWebhook": true}}'
Cost and Resource Management
Deployment Costs
Our complete test environment costs:
- EC2 t3.micro: $0.0104/hour (Free Tier eligible)
- Lambda invocations: Free (1M requests/month included)
- CloudWatch alarms: Free (10 alarms included)
- VPC endpoints: $0.01/hour per endpoint
- Total estimated cost: $0.05-$0.10 for complete testing cycle
Resource Cleanup
# Automated cleanup via CloudFormationaws cloudformation delete-stack --stack-name Starscream-DevOps-Agent-Stack
# Manual Agent Space cleanupaws devopsagent delete-agent-space \ --agent-space-id AGENT_SPACE_ID \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1
Production Deployment Considerations
Security Hardening
- Network isolation: Deploy in private subnets with VPC endpoints
- IAM boundary policies: Additional permission constraints
- Resource-based policies: Fine-grained access control
- CloudTrail integration: Comprehensive audit logging
Monitoring and Observability
- Agent Space metrics: Monitor investigation frequency and success rates
- Cost tracking: Tag-based cost allocation for DevOps Agent usage
- Performance monitoring: Track investigation response times
- Integration health: Monitor external service connectivity
Operational Procedures
- Incident playbooks: Document when to trigger manual investigations
- Escalation procedures: Define when to engage AWS Support
- Knowledge management: Maintain agent knowledge base
- Team training: Ensure operators understand agent capabilities
Availability and Next Steps
You can deploy this CloudFormation-based DevOps Agent setup today in the US East (N. Virginia) Region. The template is designed to be:
- Environment agnostic: Works in development, staging, and production accounts
- Scalable: Easily extended to multi-account architectures
- Maintainable: Clear separation of concerns and comprehensive documentation
- Cost-effective: Optimized for AWS Free Tier usage during testing
Complete CloudFormation Template
Here's the complete template that creates the Starscream Agent Space with both EC2 and Lambda test environments:
AWSTemplateFormatVersion: '2010-09-09'Description: 'AWS DevOps Agent - Starscream Agent Space with SSM Test Environment'
Resources: # DevOps Agent IAM Roles DevOpsAgentSpaceRole: Type: AWS::IAM::Role Properties: RoleName: DevOpsAgentRole-Starscream-AgentSpace AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: aidevops.amazonaws.com Action: sts:AssumeRole Condition: StringEquals: aws:SourceAccount: !Ref AWS::AccountId ArnLike: aws:SourceArn: !Sub 'arn:aws:aidevops:us-east-1:${AWS::AccountId}:agentspace/*' ManagedPolicyArns: - arn:aws:iam::aws:policy/AIOpsAssistantPolicy Policies: - PolicyName: AllowExpandedAIOpsAssistantPolicy PolicyDocument: Version: '2012-10-17' Statement: - Sid: AllowAwsSupportActions Effect: Allow Action: - support:CreateCase - support:DescribeCases Resource: '*' - Sid: AllowExpandedAIOpsAssistantPolicy Effect: Allow Action: - aidevops:GetKnowledgeItem - aidevops:ListKnowledgeItems - resource-explorer-2:Search Resource: '*'
# EC2 Instance Role for SSM (no SSH required) EC2InstanceRole: Type: AWS::IAM::Role Properties: RoleName: Starscream-EC2-SSM-Role AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: ec2.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
# VPC Infrastructure for secure testing TestVPC: Type: AWS::EC2::VPC Properties: CidrBlock: 10.0.0.0/16 EnableDnsHostnames: true EnableDnsSupport: true
# VPC Endpoints for SSM (no internet required) SSMEndpoint: Type: AWS::EC2::VPCEndpoint Properties: VpcId: !Ref TestVPC ServiceName: !Sub 'com.amazonaws.${AWS::Region}.ssm' VpcEndpointType: Interface PrivateDnsEnabled: true SubnetIds: - !Ref TestSubnet
# Lambda function that intentionally generates errors TestLambdaFunction: Type: AWS::Lambda::Function Properties: FunctionName: Starscream-DevOpsAgent-test-lambda Runtime: python3.12 Handler: index.lambda_handler Code: ZipFile: | import json import random from datetime import datetime def lambda_handler(event, context): error_scenarios = [ "Starscream: Simulated database connection timeout", "Starscream: Test API rate limit exceeded" ] error_message = random.choice(error_scenarios) raise Exception(f"Starscream DevOpsAgent Test Error: {error_message}")
# CloudWatch Alarms for incident detection CPUAlarm: Type: AWS::CloudWatch::Alarm Properties: AlarmName: Starscream-DevOpsAgent-EC2-CPU-Test MetricName: CPUUtilization Namespace: AWS/EC2 Threshold: 70 ComparisonOperator: GreaterThanThreshold Dimensions: - Name: InstanceId Value: !Ref TestInstance
LambdaErrorAlarm: Type: AWS::CloudWatch::Alarm Properties: AlarmName: Starscream-DevOpsAgent-Lambda-Error-Test MetricName: Errors Namespace: AWS/Lambda Threshold: 0 ComparisonOperator: GreaterThanThreshold Dimensions: - Name: FunctionName Value: !Ref TestLambdaFunction
Testing the Deployment
Automated Deployment Script
I created a deployment script that handles the complete setup:
#!/bin/bash# Deploy CloudFormation stackaws cloudformation deploy \ --template-file starscream-devops-agent-stack.yaml \ --stack-name Starscream-DevOps-Agent-Stack \ --capabilities CAPABILITY_NAMED_IAM
# Get stack outputsAGENT_SPACE_ROLE_ARN=$(aws cloudformation describe-stacks \ --stack-name Starscream-DevOps-Agent-Stack \ --query 'Stacks[0].Outputs[?OutputKey==`DevOpsAgentSpaceRoleArn`].OutputValue' \ --output text)
# Create and configure Agent SpaceAGENT_SPACE_ID=$(aws devopsagent create-agent-space \ --name "Starscream" \ --description "IaC-deployed Agent Space" \ --endpoint-url "https://api.prod.cp.aidevops.us-east-1.api.aws" \ --region us-east-1 \ --query 'agentSpace.agentSpaceId' \ --output text)
# Associate AWS accountaws devopsagent associate-service \ --agent-space-id $AGENT_SPACE_ID \ --service-id aws \ --configuration "{\"aws\": {\"assumableRoleArn\": \"$AGENT_SPACE_ROLE_ARN\", \"accountId\": \"$ACCOUNT_ID\", \"accountType\": \"monitor\", \"resources\": []}}"
Test Execution Results
After deployment, I executed both test scenarios:
EC2 CPU Test Results:
Starting Starscream DevOpsAgent CPU Stress TestTime: Wed Dec 17 19:31:30 UTC 2025Instance: i-0ec7728aeaa381ed8CPU Cores: 2
Starting stress test (5 minutes)...Started CPU load process 1 (PID: 6352)Started CPU load process 2 (PID: 6353)CPU load processes started for 5 minutes
Lambda Error Test Results:
StatusCode: 200FunctionError: "Unhandled"Response: { "errorMessage": "Starscream DevOpsAgent Test Error: Starscream: Test API rate limit exceeded", "errorType": "Exception", "requestId": "215d39e8-18d1-4357-8bfb-75e80a4fa669"}
CloudWatch Alarm Activation
The Lambda error alarm successfully triggered:
AlarmName: Starscream-DevOpsAgent-Lambda-Error-TestStateValue: ALARMStateReason: Threshold Crossed: 1 datapoint [1.0] was greater than the threshold (0.0)
DevOps Agent Investigation Capabilities
With the alarms triggered, AWS DevOps Agent demonstrated its investigation capabilities:
- Automatic correlation: Connected Lambda errors to recent function invocations
- Root cause identification: Recognized intentional test error patterns
- Timeline reconstruction: Showed sequence of events leading to alarm
- Contextual analysis: Understood this was a testing scenario, not production incident
- Actionable recommendations: Suggested monitoring improvements and error handling strategies
Conclusion
This Infrastructure as Code approach to AWS DevOps Agent deployment provides several key advantages:
- Reproducible environments: Consistent setup across teams and accounts
- Integrated testing: Built-in validation of agent capabilities
- Security by design: SSM-based access and private networking
- Cost-effective validation: Minimal resource usage for comprehensive testing
- Production-ready foundation: Easily extensible to real-world scenarios
The CloudFormation template creates a complete testing environment that validates AWS DevOps Agent's incident response capabilities while following AWS Well-Architected principles. This approach ensures your autonomous on-call engineer is properly configured and tested before handling real production incidents.
As organizations continue to adopt Infrastructure as Code practices, having a programmatic approach to deploying and testing AWS DevOps Agent becomes essential for maintaining reliable, scalable incident response capabilities.
Try this CloudFormation-based approach today and experience how Infrastructure as Code can streamline your AWS DevOps Agent deployment while ensuring comprehensive testing and validation.
AWS DevOps Agent is currently in preview and available at no charge with usage limits. Visit the AWS DevOps Agent documentation for the latest information and updates.
Resources and Next Steps
- Complete CloudFormation template: Available in the accompanying files
- Deployment automation script: Handles end-to-end setup
- Testing procedures: Step-by-step validation guide
- Troubleshooting guide: Common issues and solutions
- Multi-account expansion: Ready for enterprise deployment patterns
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article