Designing for Failure: Building a Highly Available Web Architecture on AWS
A practical exploration of high availability and reliability on AWS using an Application Load Balancer, Auto Scaling, health checks, and multiple Availability Zones. The article explains how these services work together to distribute traffic, detect unhealthy instances, replace failed capacity, and reduce single points of failure in a web architecture.
When I first started deploying applications on AWS, success meant something simple:
The server is running and the website is accessible.
But AWS Solutions Architect Associate concepts introduced a different question:
What happens when that server fails?
A working application is not necessarily a reliable application.
That distinction became the foundation of my next AWS project: designing and testing a highly available architecture based on the Reliability principles of the AWS Well-Architected Framework.
Instead of concentrating on configuring one EC2 server, this project focused on building infrastructure capable of detecting failures, distributing traffic, replacing unhealthy compute capacity, and continuing to operate with less dependence on any single instance.
The Problem with a Single EC2 Instance
Consider a simple architecture:
Internet → EC2 → Application
It works perfectly while the EC2 instance is healthy.
But if that instance crashes, becomes unreachable, or is accidentally terminated, the application disappears with it.
The architecture therefore contains a single point of failure.
The objective of this project was to move away from that model toward:
Users → Application Load Balancer → EC2 instances across Availability Zones
with Auto Scaling maintaining the required compute capacity.
The important change wasn't simply adding more EC2 instances.
It was changing how the infrastructure responds to failure.
Designing Across Availability Zones
One of the core reliability concepts I wanted to implement was redundancy across Availability Zones.
An AWS Region contains multiple physically separate Availability Zones.
Instead of depending on resources in only one AZ, application capacity can be distributed across multiple AZs.
Conceptually:
1
2
3
4
5
6
7
8
9
10
Internet
|
v
Application Load Balancer
/ \
/ \
v v
Availability Zone A Availability Zone B
| |
EC2 Instance EC2 InstanceThis changes the failure model.
If one application instance becomes unhealthy, another instance can continue serving requests.
And by distributing infrastructure across Availability Zones, the design also reduces dependence on a single AZ.
This was my first major lesson from the project:
High availability is not simply about having more servers. It is about avoiding unnecessary dependencies on individual resources or failure domains.
Why Put a Load Balancer in Front?
Once multiple EC2 instances exist, another question appears:
Which instance should users connect to?
Users should not need to know which EC2 instance is currently healthy.
That responsibility belongs to the architecture.
I used an Application Load Balancer (ALB) as the public entry point.
Instead of:
User → EC2
the request becomes:
User → ALB → healthy EC2 target
The load balancer distributes incoming application traffic across registered targets.
This provides something that manually creating two EC2 instances does not:
a common traffic-management layer.
Users interact with the load balancer rather than depending directly on a particular application server.
Target Groups and Health Checks
One of the most important parts of the project was understanding that a load balancer should not blindly send requests to every registered server.
It needs to know whether those servers are actually capable of serving the application.
This is where Target Groups and health checks become important.
The EC2 instances are registered as targets.
The load balancer periodically checks their health.
Conceptually:
1
2
3
4
5
ALB
|
+---- Health Check ----> EC2-A ✓
|
+---- Health Check ----> EC2-B ✓If one target becomes unhealthy:
1
2
3
4
5
ALB
|
+---- EC2-A ✗ Unhealthy
|
+---- EC2-B ✓ Healthythe unhealthy target can be removed from normal traffic routing.
This introduced an important distinction:
Running does not necessarily mean healthy.
An EC2 instance might technically exist while its web application is unavailable.
Health checks therefore evaluate whether the application endpoint is capable of responding as expected.
Auto Scaling Changed the Architecture Again
Load balancing solves traffic distribution, but another problem remains.
Suppose an EC2 instance disappears completely.
If I manually created two instances and one failed, I would eventually be left with only one.
Someone would have to notice the failure and manually create another instance.
That is where Amazon EC2 Auto Scaling changes the design.
Instead of thinking:
"I need these exact EC2 instances."
the architecture thinks:
"I need this amount of healthy compute capacity."
An Auto Scaling Group can maintain:
- minimum capacity
- desired capacity
- maximum capacity
The individual instance becomes less important than maintaining the desired state of the application fleet.
Launch Templates: Defining Replaceable Compute
For Auto Scaling to create instances automatically, AWS needs to know what those instances should look like.
That is where the Launch Template becomes useful.
The template defines important instance configuration such as the machine image, instance type, networking and security configuration, and initialization settings.
This introduced one of my favorite concepts from the project:
Servers should become reproducible and replaceable.
If an instance fails, the architecture should not depend on manually rebuilding that exact machine.
Instead, the configuration should make it possible to launch another equivalent instance.
That is a major shift from traditional server administration.
Security Groups Between Architecture Layers
The project also provided a practical way to apply least-privilege networking.
Instead of allowing the EC2 web servers to receive application traffic indiscriminately from the internet, the architecture can separate access between layers.
Conceptually:
1
2
3
4
5
6
7
8
9
Internet
|
| HTTP/HTTPS
v
ALB Security Group
|
| Application traffic
v
EC2 Security GroupThe EC2 security rules can allow application traffic originating from the load-balancer layer rather than treating every internet client as a trusted direct source.
This creates a cleaner boundary:
The ALB is public-facing.
The application instances sit behind it.
That is much closer to how I now think about cloud network design: not simply "which port should I open?" but which component actually needs permission to communicate with which other component?
The Most Important Test: Intentionally Breaking It
Creating an architecture diagram does not prove that the architecture is reliable.
So one of the most useful parts of this project was failure testing.
Instead of protecting the EC2 instances because they were "working," I intentionally treated them as disposable infrastructure.
The idea was simple:
Terminate or disrupt an application instance and observe what happens.
The expected sequence was:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
EC2 instance fails
↓
Health state changes
↓
Traffic stops being sent to the unhealthy target
↓
Remaining healthy capacity continues serving requests
↓
Auto Scaling detects lost capacity
↓
Replacement instance is launched
↓
New instance becomes healthy
↓
Capacity returns to desired stateThis test made reliability far easier to understand than simply reading its definition.
The Difference Between High Availability and Auto Scaling
Before this project, these concepts could easily sound interchangeable.
They are not.
High availability is about designing the system so that it can continue operating despite failures.
Auto Scaling manages compute capacity according to configured conditions and desired state.
Load balancing distributes traffic across healthy application targets.
Health checks determine whether those targets should receive traffic.
These capabilities complement one another.
Together they create a system that behaves very differently from a standalone EC2 server.
Reliability Is About Recovery, Not Preventing Every Failure
One of the biggest lessons from this project was that cloud architecture does not assume infrastructure will never fail.
Failures are expected.
The more useful question is:
What happens after the failure?
A fragile architecture might require:
1
2
3
4
5
6
7
8
9
10
11
Failure
↓
Administrator notices
↓
Administrator investigates
↓
Administrator creates replacement
↓
Administrator reconfigures application
↓
Service returnsA reliability-oriented architecture aims to move toward:
1
2
3
4
5
6
7
8
9
Failure
↓
Detection
↓
Traffic redirection
↓
Automated replacement
↓
RecoveryReducing manual intervention is therefore part of improving reliability.
A Cost-Conscious Learning Architecture
Another important constraint was that this was a learning project running in my personal AWS environment.
A textbook highly available production architecture can include several services and continuously running resources that generate charges.
Instead of blindly deploying every component from an architecture diagram, I treated cost as an architectural constraint.
Before creating resources, I considered:
- whether the resource was necessary to demonstrate the concept
- whether it had ongoing hourly charges
- how long it needed to remain active
- whether the same concept could be demonstrated safely with a smaller configuration
- whether it should be deleted immediately after validation
This was particularly important for services such as load balancing and other continuously billed infrastructure.
It taught me another Solutions Architect lesson:
A technically valid architecture still has operational and financial trade-offs.
Architecture is not about selecting the largest number of AWS services.
From Server Administration to Architecture
My earlier Linux/WordPress project taught me to think deeply about an individual server:
Is Apache running?
Are the permissions correct?
Can WordPress communicate with MySQL?
Which port is the service listening on?
This project introduced a different level of thinking:
What happens if the server disappears?
Where should traffic enter the architecture?
How does AWS know which instance is healthy?
How is failed capacity replaced?
How can failure in one location have less impact on the application?
That shift—from managing a server to designing a system—was one of the most valuable parts of the project.
How This Connects to AWS Solutions Architect Associate
This project turned several AWS Solutions Architect Associate concepts into something I could actually observe:
Elastic Load Balancing for distributing application traffic.
EC2 Auto Scaling for maintaining application capacity.
Availability Zones for reducing dependence on a single failure domain.
Health checks for detecting unhealthy application targets.
Launch Templates for reproducible compute configuration.
Security Groups for controlling communication between architecture layers.
CloudWatch for observing infrastructure behavior.
Most importantly, these were no longer isolated AWS services.
They became components of one reliability strategy.
Final Takeaway
The biggest lesson from this project can be summarized with one change in mindset:
Do not design infrastructure assuming nothing will fail.
Design what should happen when something does fail.
A single EC2 instance can host an application.
A reliability-oriented architecture goes further by asking how traffic is distributed, how failures are detected, how capacity is restored, and how much manual intervention is required to recover.
That is where this project became more than another EC2 deployment.
I intentionally broke part of the architecture—and the most interesting result was not the failure.
It was watching the architecture respond to it.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article