Cloud Horror Stories at AWS Community Day Malta: what the room taught me
I gave my Cloud Horror Stories talk at AWS Community Day Malta. Here are the stories I told, and the people who made the trip worth it.
On 16 October 2026, I went to Malta for the AWS Community Day to go on stage with my talk: Cloud Horror Stories: Mistakes and Anti-Patterns Not to be Repeated. It's a talk I love to deliver, because it's about mistakes made in good faith, how they got fixed, and a few laughs along the way, without pointing fingers or shaming any technology.
I had to skip the workshop day on the 15th because of work issues. I was a bit sad about that, but the conference day more than made up for it.
What I remember most is the reactions in the room. They’re the reason I share these stories in public, so someone else doesn’t have to learn the same lesson the hard way
The stories from the talk
My talk covers a few stories I have lived or witnessed, and some of them are already on my blog. Here is a short version of what I covered, with links if you are curious and want the full version.
Schrödinger's container
We were moving a web application from EC2 to containers on ECS Fargate. We deployed the development environment, and during the night, with no real user traffic, the task kept restarting. We investigated, and the memory graph pointed to the "usual" memory leak. The strange part was the trigger: the simple Application Load Balancer health check was causing the out-of-memory error, so observing the system killed it. The container was both "running" and "dead", like the famous paradox.
It turned out that the "fix" in production was a scheduled application server restart (with cron). That kept things working, but it made me think: containerizing a flawed application doesn't modernize it. Sometimes it's just a faster way to discover issues and technical debt.
"It's cloud native"
This story isn't part of the Cloud Horror Stories series on the blog yet, but I wanted to give the audience a sneak peek. I already have other stories lined up for this year, so stay tuned (or come to one of my next talks) to hear the full story live.
I was engaged on a project that, on paper, used a modern, cloud-native architecture: EKS and microservices, exactly what I love to hear when I start working on a project.
In practice, when we dove deeper into the components, the application turned out to be little more than a frontend for the on-premises infrastructure, relying entirely on a fleet of virtual and physical servers. To make things worse, the services talked to each other in ways that looked distributed, but it was really a distributed monolith: a single application cut into pieces and spread across the network.
If your Kubernetes cluster can't live without the old datacenter, and your "microservices" can't scale independently because they all depend on each other, you have a distributed monolith with extra operational cost.
Context and time explained a lot of these choices later on. What I wanted to make clear is that using a technology doesn't mean you are using an architectural pattern. You can still build a SOA architecture with plain EC2 instances!
Reflections on the us-east-1 downtime
When us-east-1 had its bad day, a lot of services that people treat as "always on" stopped working. One example that stuck with many of us was Eight Sleep: the smart mattresses suddenly became a lot less smart, and some people had trouble sleeping, or didn't sleep at all.
I already wrote my thoughts on that incident here . In the talk I went back to a few causes that keep coming up in real designs.
Some teams use us-east-1 as their main region and rely on the Route 53 console (or control-plane APIs) to perform failover, only to discover during the incident that the console wasn't available when they needed it most. Others discover that their RPO and RTO figures looked fine on a slide, but didn't match what users expected during the outage (as Canva users found out).
Unfortunately, going multi-region isn't free, and it has a lot of technical and economic implications: a lower RTO usually means higher cost and more complexity, and "architecting is a series of trade-offs" (W. Vogels, The Frugal Architect). That trade-off is fine, as long as it's a conscious one, and as long as the recovery path doesn't depend on the same control plane that just failed.
The $58,000 Bedrock bill
Yes, that bill, and the near heart attack I wrote about here: How a Billing Bug Made Me Owe $58,000 to AWS (For a Week) .
A bug in Bedrock's OpenAI-compatible API layer multiplied token accounting in a way that turned a small experiment into an expense comparable to buying a fancy car. I spotted it, documented it, reported it, and it got fixed. The scary part lasted about a week. While I was describing the issue on stage, I saw people opening their laptops (maybe to check their own Bedrock billing?).
Humor helps, but the useful takeaway is still the boring one: monitor from the application as well as from the console, and keep numbers you can understand and defend. As we learned when adopting the serverless paradigm, budget alarms are not optional if you are experimenting with AI APIs, but they don't give you full observability on your system either.
What the room gave back
Speaking about mistakes is not easy. You wonder if the story is too specific or right for the audience, and you hope people won’t remember only the huge bill and forget the lesson.
You can tell how a talk lands from the audience's reactions, and in Malta I saw plenty of them. Nodding usually means "I know that pain", and laughing means the story worked without turning into a lecture.
This means a lot to me, because understanding someone else's mistake is often more useful than seeing a perfect architecture in a keynote.
After the talk I had some interesting conversations over coffee and snacks (organizers: you nailed the refreshments). Someone had seen a similar health-check problem, someone else was thinking about their failover strategy, and others were just curious about the stories I didn't have time to tell.
If even one person who attended prevents an incident, then preparing the slides and squeezing the trip into my work calendar were all worth it.
The community
The talk was only one part of the day. Community Days work because of everything that happens around the talks.
The venue was the Hard Rock Hotel, so new that I got lost in the mall because there were almost no signs for the hotel yet. Luckily, I met other attendees in the same situation, and we figured it out together. It's a small detail, but it sums up the vibe well: in a community, you're never lost alone.
I also want to give credit to the organizers for the amazing job they did with such a small group of volunteers. The day before, some of the rooms were still under construction; the next day, almost by magic, everything was ready for the audience and the speakers. You don't usually perceive that effort when you only see the final result, but when you talk to the organizers they tell you everything they had to fix and set up to make it work (and their stories are precious advice for other Community Day organizers).
Among the people I want to mention:
Kenneth Attard, one of the organizers and an AWS Hero with a networking superpower. Events like this don't happen without people who care about the details and about the community around them.
Simon Muskaat, who took care of all the speakers, from the schedule to a document with details about power adapters, routes to and from the airport, and hotel advice. Keeping everything under control and accommodating the needs of 44+ people can be really stressful. Thank you!
Davide De Paolis: I had already had the pleasure of co-speaking with him at AWS Community Day Netherlands, and it's always good to meet again in a different country and pick up the conversation where we left off. By the way, he's a Community Builder and a super active speaker: if you've been to a community event, chances are you've already seen one of his talks.
Vadym Kazulkin, AWS Hero. I had already met him at Community Day Rome and loved his talk there, so it was great to cross paths again in Malta and chat at the speakers' dinner.
Some of the best moments aren't even on the agenda: drinks on the terrace after the event, the speakers' dinner, and a last-minute, self-organized dinner on a local attendee's advice, which we gladly followed. Add the conversations with speakers, organizers, and attendees, the familiar faces from other Community Days, and the people I had only seen online or read on blogs before.
We also ended up with a group chat that is still active, and for me that's one of the best signs that an event worked.
When we said goodbye after dinner, the phrase I heard most was "See you in Vegas". I can't wait to be there.
Why you should attend Community Days
I've already written about why you should become an AWS Community Builder , and I don't want to repeat the list of program benefits here. You don't need to be part of a program to be part of the community.
Malta reminded me of something much simpler. Community is the person who helps you find the venue when the hotel still has no signs or the speaker who gives you feedback so your talk is better next time.
Community is also the dinner where you talk about Route 53 failover and Bedrock bills without anyone thinking about business or job titles.
You can learn a lot from documentation and blog posts, but you learn more when someone in front of you says "we did this, it broke, and here is what we changed" (or asks you for advice).
I've seen this at User Groups in Italy and at Community Days in other countries. The language and the venues change, but there are always people who care and show up to share their experiences.
That's why I keep applying to speak, helping organize User Groups, and showing up as an attendee even when I'm not on the agenda!
If you are hesitating about going to a community event, my advice is to go without thinking further about it, and if you have a story to tell, consider telling it. You don't need a perfect talk: you’ll improve it over time, but first you have to accept it won’t be perfect at the beginning.
If you were in the room in Malta: thank you. If you weren't, I hope to see you at the next event near you, and if you want to share your experience, drop a comment below or find me on LinkedIn. I'm always happy to compare scars :)
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article