The night S3 versioning saved our data project

The night before our data analysis project was due, I was cleaning up the S3 bucket our group used and ran aws s3 rm --recursive on the wrong prefix. The folder with our cleaned dataset, the one that took two of us a whole weekend to produce, vanished from the console.

I'm not going to pretend I stayed calm about it.

About a month earlier I had turned on S3 Versioning  on that bucket, mostly because a study guide said it was a good idea. With versioning on, a normal delete doesn't actually remove the object. S3 adds a delete marker as the newest version. The object looks deleted, but the older versions are still there underneath.

When I switched on Show versions in the console, every file was there with a delete marker sitting on top of the real data.

Deleting a delete marker makes the previous version current again. Doing that by hand for a whole folder would have taken forever, so I scripted it with the CLI:

Ten minutes later the folder was back exactly as it had been. We submitted on time and I did not tell my group how close it was until after grades came out.

Versioning isn't free, since every old version is stored and billed. So afterward I added a lifecycle rule that permanently removes noncurrent versions after 30 days and cleans up expired delete markers. That keeps a safety window without the bucket growing forever.

It's also worth knowing what versioning doesn't protect against: someone deliberately deleting specific versions. For more important data there's MFA delete or Object Lock. For a class project, versioning plus a lifecycle rule was exactly enough. It's the first setting I turn on in every new bucket now.