AWS Builder Center

The Night I Deleted Every User Photo with a Single CLI Command

One mistyped bucket name, a missing S3 safety net, and 4,000 user photos turned to dust. A 2am cloud horror story and the tiny habits that now stop me from ever running `aws s3 rm --recursive` without triple-checking.

It was 11:47 PM. I was three cups of chai deep, ready to call it a night. All I had to do was clean up a few hundred test images from a dev S3 bucket. I'd done it a hundred times. A quick  aws s3 rm , then bed.
The plan: set an environment variable to  club-photos-dev , run the delete, and sleep like a responsible engineer. My shell history already had the command waiting. I typed:
export BUCKET=club-photos-prod
aws s3 rm s3://$BUCKET --recursive
I hit Enter without looking. The terminal began to scroll — fast. Key after key flew past. At first I thought, "Wow, that's a lot of test photos." Then I saw it: uploads/2026/summit/group_photo_final_final.jpg. A real photo. An actual user memory.
My stomach hit the floor. I bashed  Ctrl+C , but the damage was done. 4,000+ images — every photo our college club's members had ever uploaded — were gone. The bucket was empty. No recycle bin. No version history. Just a blinking cursor that seemed to mock me.
I opened the S3 console, hoping for a miracle. Versioning: Not enabled. I had never turned it on because "it's just a small project." What were the odds of a catastrophic typo at midnight? The universe had just answered.
No backup. No cross-region replication. No soft delete. Just a hollow  /uploads/  prefix returning  404  on every image link. I messaged the club president with the understatement of the year: "Small issue with the gallery. Working on it."
I didn't sleep that night. The next morning, I had to tell a dozen angry users their photos were gone forever. We posted an apology. I rebuilt the gallery with placeholder graphics. And I spent the entire weekend implementing guardrails I should have had from day one.



What that night taught me (and what I now do religiously)

  • Turn on S3 Versioning everywhere. Even on dev. It costs pocket change and would have let me restore everything in three clicks.
  • Use MFA Delete on production buckets. That extra five-second token check would have saved 4,000 files and my dignity.
  • Never run destructive CLI commands without a dry-run first. S3 doesn't have a native  --dryrun , but a simple  aws s3 ls --recursive  would have shown me I was about to nuke real data.
  • Isolate permissions. My cleanup script now runs under a role that can only touch a specific prefix inside the dev bucket. A typo in the bucket name now gets me an "Access Denied," not an extinction event.
  • No risky commands after 10 PM. Tired brains make irreversible mistakes. I now enforce a personal rule: destructive actions get a second pair of eyes or wait until morning.
The  aws s3 rm  command doesn't ask "Are you sure?" It trusts you completely. And if you hand it a production bucket by mistake, it will faithfully, silently, and instantly delete everything you point at it. I learned that the hard way.
Now every bucket I touch has versioning, MFA delete, and an IAM boundary that would make a security auditor proud. And every time I see  s3://  in a terminal, I pause just long enough to remember that night — and triple-check the name. Sometimes the cheapest lessons in the cloud are the most expensive ones.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article