
OmniTrace
"Our comprehensive technical project breakdown, architecture topology, and development logs are completely documented here in our open-source production repository." Link : https://github.com/likhilliki/OmniTrace
π OmniTrace β Autonomous Cloud Incident Triage & Self-Healing Pipeline
Track 2: Ship It β AWS Serverless Hackathon submission Powered by Amazon Bedrock Nova Pro, Step Functions, EventBridge, Lambda, DynamoDB, and API Gateway
ο»Ώ
ποΈ Architecture Overview
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β AWS Cloud (us-east-1) β β β React SPA β API Gateway (HTTP API v2) β (Amplify) ββPOSTβββΊ β /api/trigger βββΊ ApiHandler Lambda β ββGETβββ β /api/incidents βββ ApiHandler Lambda β β β β² β β β put_events β query/scan β β βΌ β β β EventBridge Default Bus DynamoDB β β source: omnitrace.alert OmniTrace_Incidents β β β β² β β β StartExecution β PutItem/UpdateItem β β βΌ β β β βββββββββββββββββββββββββββββββββββββββ β β β Step Functions State Machine β β β β β β β β ββββββββββββ β β β β β AUDITOR β βββ Bedrock Nova Pro β β β β β Agent β (root cause + INR) β β β β ββββββ¬ββββββ β β β β β β β β β ββββββΌββββββ β β β β β PATCHER β βββ Bedrock Nova Pro β β β β β Agent β (remediation cmds) β β β β ββββββ¬ββββββ β β β β β β β β β ββββββΌβββββββ β β β β β VALIDATOR β βββ Bedrock Nova Pro β β β β β Agent β (safety audit) β β β β ββββββ¬βββββββ β β β β β β β β β ββββββΌβββββββββββββββββββββββ β β β β β Choice State β β β β β β verdict == EXECUTE ? β β β β β ββββ¬βββββββββββββββββββββ¬ββββ β β β β VETO EXECUTE β β β β β β β β β β EscalateToSRE SelfHealer Lambda β β β β (manual review) (mock remediation) β β β βββββββββββββββββββββββββββββββββββββββ β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββο»Ώ
β AWS Services Utilized
Service | Purpose |
|---|---|
Amazon Bedrock (Nova Pro) | AI agents for root-cause analysis, command generation, and safety validation |
AWS Step Functions | Orchestrates the 3-agent pipeline with Choice state for VETO/EXECUTE routing |
Amazon EventBridge | Receives Β omnitrace.alertΒ events and triggers the state machine |
AWS Lambda (Python 3.12) | Agent Dispatcher, Self-Healer, API Handler |
Amazon DynamoDB | Persists all incident records, agent reasoning trails, and resolution logs |
Amazon API Gateway (HTTP API v2) | REST API consumed by the React SPA |
AWS Amplify | Hosts the React + Vite + Tailwind CSS frontend |
Amazon CloudWatch | Logs, metrics, and alarms for the full pipeline |
AWS IAM | Least-privilege roles for every Lambda and the Step Functions state machine |
ο»Ώ
π Project Structure
omni-trace/βββ template.yaml # AWS SAM / CloudFormation templateβββ samconfig.toml # SAM CLI deployment configurationβββ backend/β βββ handlers/β βββ agent_dispatcher.py # AUDITOR + PATCHER + VALIDATOR agents (Bedrock)β βββ api_handler.py # REST API (GET /incidents, POST /trigger)β βββ self_healer.py # Executes remediation, marks RESOLVEDβ βββ requirements.txt # Python dependenciesβββ frontend/β βββ index.htmlβ βββ package.jsonβ βββ vite.config.jsβ βββ tailwind.config.jsβ βββ postcss.config.jsβ βββ .env.exampleβ βββ src/β βββ main.jsxβ βββ index.cssβ βββ App.jsx # Full SRE dashboard SPAβββ cdk/ # Optional CDK stack (pre-existing)βββ README.mdο»Ώ
π Deployment Guide
Prerequisites
# Install AWS SAM CLIpip install aws-sam-cliο»Ώ
# Install Node.js 18+ for frontend# Ensure AWS credentials are configuredaws sts get-caller-identity1οΈβ£ Enable Amazon Bedrock Model Access
In the AWS Console β Amazon Bedrock β Model access:
- Enable Amazon Nova Pro (Β
amazon.nova-pro-v1:0Β)
2οΈβ£ Backend Deployment (SAM)
# Clone and enter repocd omni-traceο»Ώ
# Build all Lambda functionssam buildο»Ώ
# Deploy to AWS (interactive β fill in your S3 bucket and stack name)sam deploy --guidedο»Ώ
# On subsequent deploys:sam deploySAM deploy will output:
Outputs:ApiEndpoint = https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prodDynamoDBTable = OmniTrace_IncidentsStateMachineArn = arn:aws:states:us-east-1:...β οΈ Copy the ΒApiEndpointΒoutput β you need it for the frontend.
3οΈβ£ Frontend Deployment (AWS Amplify)
Option A β Amplify Console (Recommended)
- Push this repository to GitHub / CodeCommit / Bitbucket
- Go to AWS Amplify β New App β Host web app
- Connect your repository and branch
- Set build settings:
version: 1frontend: phases: preBuild: commands: - cd frontend - npm ci build: commands: - npm run build artifacts: baseDirectory: frontend/dist files: - '**/*' cache: paths: - frontend/node_modules/**/*- Add environment variable in Amplify console:
- Key: Β
VITE_API_URLΒ
- Value: Β
https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prodΒ
- Save and deploy.
Option B β Local development
cd frontendcp .env.example .env.local# Edit .env.local and set VITE_API_URL to your ApiEndpointο»Ώ
npm installnpm run dev# Visit http://localhost:3000Option C β Manual S3 + CloudFront
cd frontendnpm installnpm run buildο»Ώ
# Create S3 bucket for hostingaws s3 mb s3://omnitrace-dashboard-$(date +%s)ο»Ώ
# Upload buildaws s3 sync dist/ s3://YOUR_BUCKET_NAME --deleteο»Ώ
# Or use AWS CLI to create CloudFront distributionο»Ώ
π§ Local Testing
Test individual Lambda handlers locally:
# Test API Handlersam local invoke OmniTraceApiHandler --event events/trigger.jsonο»Ώ
# Test Agent Dispatcher (AUDITOR role)sam local invoke OmniTraceAgentDispatcher --event events/auditor.jsonο»Ώ
# Start local APIsam local start-api --port 8080Sample event files:
Β
events/trigger.jsonΒ { "requestContext": { "http": { "method": "POST" } }, "rawPath": "/api/trigger", "body": "{\"triggeredBy\": \"test\"}"}Β
events/auditor.jsonΒ { "agent_role": "AUDITOR", "incident": { "incidentId": "INC-TEST-001", "service": "omnitrace-api", "errorType": "MEMORY_EXHAUSTION", "severity": "P1", "region": "us-east-1", "errorLogs": "[ERROR] Lambda runtime exited: signal: killed\n[CRITICAL] Memory 512/512 MB exhausted", "timestamp": "2025-01-15T14:30:00Z" }}ο»Ώ
π Key Design Decisions
Decision | Rationale |
|---|---|
Nova Pro model | Optimal for structured JSON output, strong reasoning at low cost |
Bedrock Converse API | Unified API across models, supports system prompts cleanly |
3-agent sequential pipeline | Separation of concerns β each agent specializes in one task |
VETO safety gate | Prevents autonomous execution of destructive operations |
DynamoDB for agent state | Full audit trail of every reasoning step, queryable by incidentId |
EventBridge as trigger | Decouples simulation from pipeline β real CloudWatch alarms can fire the same event |
HTTP API v2 | Lower latency and cost than REST API for simple CRUD operations |
INR cost estimates | India-first audience, more relatable for the hackathon demographic |
ο»Ώ
π‘οΈ Security Notes
- All Lambda functions use least-privilege IAM roles
- Bedrock model access is scoped to specific model ARNs
- DynamoDB access is scoped to the single OmniTrace_Incidents table
- API Gateway has CORS configured (restrict Β
AllowOriginsΒto your Amplify domain in production)
- VALIDATOR agent acts as a mandatory safety gate before any auto-remediation
ο»Ώ
π Extending OmniTrace
- Connect real CloudWatch Alarms: Replace the UI trigger with an EventBridge rule matching Β
aws.cloudwatchΒalarm state changes
- Add SNS notifications: Subscribe an SNS topic to send escalation emails when VALIDATOR issues VETO
- Multi-region support: Deploy the stack to multiple regions and use EventBridge cross-region targets
- Slack integration: Add a Lambda step after VETO to post to Slack with an approve/reject button
- Historical analytics: Add a Glue + Athena pipeline over DynamoDB exports for incident analytics
ο»Ώ
π License
MIT β Built for the AWS Serverless Hackathon, Track 2: Ship It.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article