AWS Builder Center
My deployment succeeded. My app didn't. So I built a tool that asks why.

My deployment succeeded. My app didn't. So I built a tool that asks why.

A one-port typo passed CloudFormation and then broke my app. For my first AWS project, I built ADI: a tool that traces what a change will break before you deploy, then checks the prediction against CloudWatch afterwards.

What I built for the WeMakeDevs x AWS First Commit hackathon, and what my first real AWS project taught me.

The bug that started it

Before this hackathon, I had never deployed anything to AWS. So I did what most students do: I read about CloudFormation, wrote a template, and deployed a small app. It had a load balancer, two containers on ECS Fargate, and a PostgreSQL database on RDS.
Then I tried something small. I changed one number in the template: the port on the database's security group, from 5432 to 5433. A typo, basically.
Here is what happened:
  • The change set looked harmless. It listed the resources to modify, with no warning.
  • CloudFormation finished in about eleven seconds and said  UPDATE_COMPLETE .
  • About twenty seconds later, every request to my app returned an error.
From CloudFormation's side, nothing was wrong. It changed the security group exactly as I asked. The problem was one step further along. My containers reach the database through that security group, on port 5432, and I had just closed that door on them.
What surprised me most was that the answer was already in my template. The security group protects the database. The task definition reads the database's address. The service runs that task definition. The whole chain was written down; no tool followed it for me.
That became our project idea.

What we built: ADI

ADI (AWS Deployment Intelligence) answers one question before you deploy:
If I deploy this change, what else can it break?
The idea I kept coming back to was a row of dominoes. When you change one resource, which others fall with it?
ADI works in five steps:
  1. Map the dependencies. It reads the current and the proposed template and builds a graph of what depends on what, from every  Ref ,  GetAtt ,  Sub  and  DependsOn . Each link gets a meaning, such as "protected by", "runs" or "routes to".
  1. Find the changes. It compares the two templates and labels each resource as created, updated, replaced or deleted. It also knows which changes force AWS to rebuild a resource, like renaming a database, which deletes the old one along with its data.
  1. Follow the dominoes. From each changed resource, it walks the graph to everything that depends on it.
  1. Apply the rules. Six rules turn that into findings, each with a severity, the chain of dominoes, and a recommendation. Every fact in a finding says where it came from: the diff, the graph, the template, the change set, or an AWS documentation page.
  1. Check itself afterwards. Each finding lists the signals that should move if it is right. After you deploy, ADI reads those signals from CloudWatch and reports whether the prediction held.
For my port typo, ADI reports HIGH: Service loses network access to Database, with the full path from the security group to the service. It even cites the AWS documentation that explains the twenty second delay: security groups don't cut connections that are already open, so the app only fails as its connections refresh.


Rules decide, AI explains

I wanted to use Claude on Amazon Bedrock, but I was careful about what it is allowed to do. If a tool warns you that a change is dangerous, you should be able to see exactly why, and get the same answer every time. So in ADI:
  • Rules make every finding. They are deterministic, so the same input always gives the same result.
  • Claude only explains them in plain language. If its answer mentions a resource that isn't in the finding's evidence, ADI rejects the answer.
  • If Claude is unavailable, the dashboard says so plainly and still shows the findings. It never shows a made-up paragraph.

How it runs on AWS

Service
What it does in ADI
Lambda (Node 22, arm64)
Runs the analysis and verification, only when called
API Gateway
The API the dashboard talks to
DynamoDB
Stores every analysis
Amplify Hosting
Hosts the React dashboard
CloudFormation
The input ADI analyzes, as templates or change sets
CloudWatch
The metrics and logs used to verify predictions
Amazon Bedrock
Claude writes the plain language explanations
IAM
ADI's role can only read the one stack it analyzes
One cost decision I'm happy with: the demo app has no NAT gateway. A NAT gateway alone would have cost more than everything else combined, so the whole demo stack runs for about US$0.10 an hour.
I also added a GitHub Actions workflow. When a pull request changes a CloudFormation template, it runs the same engine and posts the findings as a comment. That way the warning shows up in code review, before anyone deploys.


What fought back

My own verification had bugs. I wanted ADI to prove its predictions, not just make them. So it compares the fifteen minutes before a deployment with the minutes after. My first live run said MATCHED, but the "before" window showed three errors a minute from an app I knew was healthy. It took me a while to find why: CloudWatch labels each minute of data by when that minute starts. So the minute the deployment happened in, half healthy and half broken, was counted as "before". Now ADI only compares whole minutes, and the baseline came out as a clean zero.
A day later I found a second one. When I re-checked an old result, its "after" window ran past the moment I had restored the app, so it mixed the broken period with the fixed one. Now the window ends where the next deployment starts.
I was wrong about my own test. I had written in my docs that one scenario, halving the containers' memory, would come out UNCONFIRMED. When I actually ran it, memory use rose from 2.7% to 3.7%, which is exactly what ADI predicted, so it said MATCHED. My assumption was wrong, not the tool: memory use is a percentage, so halving the memory raises it even when the app uses the same amount. I corrected my docs. If I say ADI is evidence based, my own documentation has to follow the evidence too.
Setting up a brand new AWS account. This was all new to me:
  • I typed Hyderabad ( ap-south-2 ) instead of Mumbai ( ap-south-1 ) into  aws login  and spent a while confused by a token error.
  • I found out Claude isn't offered on Bedrock in every region, so ADI calls it in US East while everything else runs in Mumbai.
  • My new account couldn't call any Bedrock model at first. AWS support traced it to a problem with my payment mandate, and the access review is still in progress as I write this. Luckily, ADI was designed so the findings never depend on the AI step.

Results on real AWS

I ran the scenarios against the live stack and broke it on purpose.
Test
What ADI predicted
What CloudWatch measured
Verdict
Database port typo, through a change set
HIGH: the service loses the database
Connections 2 → 0, errors 0 → 291 a minute, log errors 0 → 246 a minute
MATCHED
Secret permission removed
HIGH: new containers can't start
Nothing moved, because running containers already had the secret
UNCONFIRMED
Memory halved
MEDIUM: memory use will rise
Memory 2.7% → 3.7%
MATCHED
The UNCONFIRMED result is my favourite. The risk was real, but nothing had failed yet, and ADI said exactly that instead of claiming a match.

Built as a team

ADI was a team effort by four students:
  • Gowtham K (me): led the team and worked across every part of the project, including the research.
  • Jaya Suriya T R: Research and Development.
  • Pranav A S: Design, icons and resources.
  • Kannan K: Security and deployment.

What I learned

  • A successful deployment is not the same as a working app.
  • Most of the information you need to predict a failure is already in your template.
  • Testing against real infrastructure finds bugs that unit tests don't. I had over 140 tests passing, and the live runs still found two bugs in my verification logic.
  • Writing down what actually happened, including where I was wrong, made the project better.

Try it

I built ADI with Claude Code as my AI pair programmer, and every commit it helped with is marked in the repository history. Thanks to WeMakeDevs and AWS for the push to ship my first real project on AWS.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article