AWS Builder Center

I Built a Broken AWS Account on Purpose. Here's What It Taught Me About Detection.

Building a CSPM scanner from scratch: how I set up an isolated sandbox, threat modeled six CIS controls before writing checks, and provisioned a deliberately broken environment to test against.

Security Hub already tells you when an S3 bucket is public. So why would anyone write their own scanner?
Because reading a finding and producing one are different skills. When a managed service hands you a list of misconfigurations, you learn to triage. When you write the check yourself, you learn why the finding exists, what it misses, and how many ways a control can be technically satisfied while the underlying risk stays wide open.
I'm a cybersecurity master's student, and I've been building an AWS security posture monitor from scratch as a portfolio project. This article covers the first three units of that build: setting up a sandbox that can't hurt me, threat modeling before writing any code, and deliberately breaking an environment so I'd have something real to detect.

Step 1: Build the account you can afford to break

The first thing I wrote wasn't code. It was an account setup document.
If your plan is to provision a public S3 bucket and a security group open to  0.0.0.0/0  on port 22, you do not want that living next to anything you care about. I used a dedicated sandbox account with a hard separation between what runs the lab and what scans it:
  • Root user locked behind MFA and then left alone entirely.
  • An admin IAM user ( annafi-cli ) with no console access. CLI only. This is what applies Terraform.
  • A read-only scanner role with the AWS-managed  SecurityAudit  policy attached, assumed through a profile chain. The scanner cannot change anything. If my code ever tries, it gets an  AccessDenied  and I learn something.
  • Budget alerts, because a lab that quietly costs money is a lab you'll stop running.
That last split is the one worth stealing even if you build nothing else from this article. Separating the identity that creates resources from the identity that reads them means your scanner's blast radius is provably zero. It also forces you to confront credential chaining early, which is a real AWS skill that no tutorial makes you practice.
Practical note: verify the chain works before you build anything on top of it. A quick  sts get-caller-identity  against the scanner profile should return the scanner role's ARN, not your admin user's. I've seen enough half-configured profile chains to know this is worth ten seconds.

Step 2: Threat model before you write checks

The temptation at this point is to open the CIS AWS Foundations Benchmark, pick every control that looks interesting, and start writing scanner functions. I did something slower instead, and it changed the whole shape of the project.
I wrote an architecture document that answered four questions before a single check existed:
What am I actually defending against? I ran a STRIDE pass over the account. Spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege. Not because STRIDE is magic, but because it forces you to name the adversary. "Public S3 bucket" isn't a threat. "An unauthenticated internet user reads customer data because the bucket policy allows  * " is a threat, and it tells you what the check needs to prove.
Which controls, and why those? I scoped to six controls from CIS AWS Foundations v7.0.0 rather than trying to cover the whole benchmark. Six is enough to hit multiple services and multiple failure modes. The full benchmark is enough to guarantee you never finish. Every control I picked, I wrote down a one-line justification for. If I couldn't justify it, it got cut.
Where does the scanner stop? An explicit scope boundary. My scanner reads configuration state. It does not attempt exploitation, it does not cross account boundaries, and it does not evaluate application-layer risk. Writing this down prevented a lot of scope drift later.
What does severity mean here? This is the one most people skip. If your scanner emits HIGH and MEDIUM findings, you need a defensible rule for which is which, or your output is just vibes with a color code. I anchored severity to exposure and blast radius: is the resource reachable from outside the account, and what does compromise of it grant? A public bucket containing nothing still scores differently from an IAM policy granting  *:* .
The architecture doc took longer than I expected and made everything after it faster. It's also the artifact I'd point a hiring manager at first, because it's the part that shows judgment rather than syntax.

Step 3: Build the vulnerable lab first

Here's the ordering insight that surprised me: I built the broken environment before I built the scanner.
It sounds backwards. It isn't. A detection you can't test is a detection you don't have. If you write the scanner first, your only feedback loop is "it didn't crash," which is not the same as "it works." Building the vulnerable lab first means every check you write has a known-bad target to prove itself against, and a clean baseline to prove it isn't producing false positives.
My Terraform lab provisions, deliberately:
  • A public S3 bucket with permissive access settings
  • An unencrypted S3 bucket, to catch the difference between "exposed" and "unprotected"
  • A security group allowing SSH from anywhere, inside an isolated lab VPC
  • An IAM role with a  *:*  policy, attached to an EC2 trust relationship but left inert
  • An unencrypted EBS volume
  • CloudTrail configured in a single region, with a log bucket set to  force_destroy 
Two details in there matter more than the resource list. First, everything lives in a dedicated lab VPC, so the network exposure is contained. Second, the CloudTrail log bucket is  force_destroy , which means  terraform destroy  actually completes instead of failing on a non-empty bucket and leaving you with orphaned resources. Small thing. Saves real frustration.
The  *:*  IAM role deserves its own note. It's attached to an EC2 trust policy but nothing assumes it. That's intentional: I wanted a maximally over-permissive policy that carries no actual live risk in the account. The check that finds it doesn't care whether anything is using it, and that's exactly the point. Unused over-permissive roles are a genuine finding in real environments, not a lab artifact.

What I'd tell someone starting this

Write the setup doc. Not for anyone else. For you, in three weeks, when you can't remember which profile assumes which role.
Pick fewer controls than you want to. Depth on six beats a shallow pass on sixty, and nobody reviewing your work is counting.
Make the scanner read-only at the IAM layer, not just by convention. "My code doesn't call any write APIs" is a claim. A role without write permissions is a proof.
Have an opinion about severity and write it down. This is the single clearest signal that separates a project that ran an SDK loop from one that thought about risk.
Build the broken thing first. You cannot validate a detector without a known-bad case, and inventing the known-bad case after the fact tends to produce detectors that only catch the thing you already imagined.

Where this goes next

The next unit is the design-heavy one: the check interface and the finding data model. Every check needs to emit the same shape, and getting that shape right early determines whether adding check number twenty is a five-minute job or a refactor. What is a finding, minimally? A control reference, a resource identifier, a severity, evidence for why it fired, and enough context to remediate. Getting that wrong is how scanners end up with output nobody can act on.
I'll write that one up once it's built.


The full project, including the Terraform lab and architecture docs, is on GitHub:  https://github.com/DevAnnafi/AWS-Security-Posture-Monitor\ *. I'm always interested in how other people scoped their first detection project, so if you've built something similar, I'd like to hear what you'd do differently.*
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article