Stop Thinking in AWS Services: Start Thinking in Systems

By Madni Sagheer, Software Engineer

[embedded content: hero · services to system]

Most of us learn AWS backwards. You pick up EC2, then S3, then Lambda, and by the time you start a real project you're basically a person with a hammer collection looking for nails. I did this for a long time.

The thing that finally clicked for me is boring: knowing what a service does tells you nothing about whether you need it.

So now, before I touch any service names, I try to answer some plain questions. What does the thing actually have to do? What breaks first? What really needs to scale, and what's fine staying small? What am I willing to give up? Only after that do I sketch boxes, and the AWS names go on the boxes last. If I catch myself saying "we'll use Lambda" in the first five minutes of a design chat, that's usually a sign I skipped something.

My rough order of thinking looks like this:

Before picking Lambda, I need to understand the workload. Before picking a database, I need to know how the data will be read and written. Before adding a queue, I should ask whether anything here really needs to be asynchronous. Before adding a cache, I should know what is actually slow.

[embedded content: design sequence · 10 steps]

There's almost never one perfect design. Simple and highly scalable usually pull in opposite directions. A small app doesn't need a distributed architecture, and every extra component is something someone has to run, monitor and debug at some point.

Managed services work the same way. They take a lot of operational work off your plate, but sometimes you need more control over the runtime, the networking or the deployment setup. Neither choice is wrong. It depends on what you're building.

Take a basic case. A user submits a form, and we need to store it, process a document and send a notification. The user wants to know we got it. They don't want to stare at a spinner while a PDF gets chewed through.

So the request path is just:

and the slow stuff goes off to the side:

Notice I haven't said a single AWS product. That's on purpose. What matters is why each piece exists. The application handles the request. The database holds state. The queue keeps slow work away from the user. The worker can fail and retry without anyone noticing. Storage holds files, and the notification piece handles communication.

Once that makes sense, then fine: pick API Gateway, Lambda, ECS, EC2, RDS, DynamoDB, SQS or S3, whatever fits.

[embedded content: reference architecture · request and background paths]

Lambda is great for short, bursty, event-driven jobs. I like it. But a long-running process, something with an unusual runtime, or a service that's just always busy anyway? A container is often simpler, cheaper, and honestly less annoying to debug.

"Should I use Lambda?" is a bad question. "What does my workload look like, and which compute model fits it?" is the one to ask.

Here's a fully synchronous flow:

If one thing in that chain gets slow, the whole request gets slow. If one fails, the user sees a failure.

Push the right work behind a queue and it changes:

A worker picks the task up later, and your request stops depending on five other systems being healthy, which is lovely. But now you own retries, duplicate messages, dead-letter queues, visibility timeouts and monitoring, and you'd better make your worker idempotent or you'll process the same thing twice.

[embedded content: synchronous vs asynchronous design]

The happy path is easy. What happens when the database blips for a minute, the third-party API just hangs, or the queue starts growing and nobody notices?

Timeouts, retries, exponential backoff, idempotency and dead-letter queues aren't polish you add in week six. They're how the system behaves when things go wrong, which in production is often.

I ask myself: if this breaks at 2 AM, how will I even know why?

Logs, metrics, traces, alerts, and a request or correlation ID that follows a call across services. CloudWatch does a lot of this on AWS, but the habit matters more than the tool.

For security, I try to think "what should this component be allowed to do?" rather than "which security service do I turn on?" Who needs access? Which permissions are truly needed? Where do secrets live? What stays private? What gets encrypted? Least privilege is something you design in, not an IAM tweak you do at the end.

Cost works the same way. Oversized instances, infrastructure running all night for nobody, logs that never expire, S3 buckets full of old files. All of it adds up quietly. That said, saving twenty dollars a month while making the system miserable to run is a bad trade too. So is building a hugely scalable setup for something three people use.

[embedded content: failure, observability, security, cost]

Instead of "I want to use Lambda," try: "I have short-lived, event-driven work with variable traffic. Is Lambda a good fit?"

Instead of "I need DynamoDB," try: "What are my data access patterns, and does a key-value or document database fit them?"

Instead of "I need SQS," try: "Do I need to decouple these components and process this work asynchronously?"

It's a small change in wording, but it changes how you design.

AWS is a giant toolbox and you don't get points for using more of it. A diagram with fourteen services looks impressive in a presentation and is a pain at 2 AM. I'd take the boring design I can explain to a new teammate in ten minutes.

So next time you start a design, don't open the services list. Start with the system. Understand the problem. Identify the trade-offs. Then choose the services.