What Really Happens When an AWS Data Center Dies
A layer-by-layer look at AWS Multi-AZ: load balancer health checks, Auto Scaling replacement, and RDS failover, with a diagram for each.
Series: How to complete certification... (5 articles)
- …
- 5What Really Happens When an AWS Data Center Dies This article
Level: Intermediate  | Reading time: 9 minutes  | Topics: VPC, Availability Zones, ALB, Auto Scaling, RDS Multi-AZ
A data center can lose power, catch fire, or get cut off by a network fault. AWS designs around that fact instead of hoping it never happens. This is a tour of exactly what keeps an application alive when one of its data centers goes dark, with a diagram for every layer.
1. Regions and Availability Zones, Precisely Defined
Every AWS Region is a cluster of Availability Zones (AZs), and every AZ is one or more physically separate data centers with independent power, cooling and networking. They sit close enough for single-digit-millisecond links between them, but far enough apart that the same flood, fire or grid failure should not take out two at once.
1
2
3
4
5
6
7
8
9
10
11
REGION: ap-south-1 (Mumbai)
+--------------------------------------------------+
| |
| AZ-A (ap-south-1a) AZ-B (ap-south-1b) AZ-C |
| +-------------+ +-------------+ +------+ |
| | data center | | data center | | ... | |
| | own power | | own power | | | |
| | own cooling | | own cooling | | | |
| +-------------+ +-------------+ +------+ |
| \_____________ low-latency fiber _________|
+--------------------------------------------------+A Region is a geography (Mumbai, Frankfurt, N. Virginia). An AZ is a failure domain inside it. Most services that matter for uptime are built around spreading work across at least two AZs, never relying on just one.
The rule this whole article is built onNothing that can fail should exist in only one place. Compute, data and the path between them all need a twin in a different AZ.
2. The Three Layers That Have to Survive Together
High availability is not one AWS feature, it is three layers that each solve a different failure, all working at once.
1
2
3
4
5
LAYER WHAT IT SURVIVES AWS SERVICE
---------------- ----------------------- -----------------------
1. Traffic entry An AZ's network is gone Elastic Load Balancer
2. Compute A server or AZ dies Auto Scaling Group
3. Data A database AZ dies RDS Multi-AZIf only one layer is covered, the application still goes down. A load balancer spread across AZs is useless if every server behind it lives in one AZ. The next three sections open each layer separately.
3. Layer One: The Load Balancer Never Lives in One Place
An Application Load Balancer (ALB) is not a single server. AWS runs its nodes in every AZ you attach it to, and each node only forwards traffic to healthy targets.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Users
|
v
+-----------------------------+
| Application Load |
| Balancer (ALB) |
| node in AZ-A | node in AZ-B |
+---------------+---------------+
| |
health checks health checks
every N seconds every N seconds
| |
v v
+--------------+ +--------------+
| EC2 target | | EC2 target |
| AZ-A | | AZ-B |
+--------------+ +--------------+The health check is what makes this real, not theoretical. By default the ALB pings each target on an interval (commonly every 30 seconds) and needs a set number of consecutive failures (commonly 2 to 5) before it marks a target unhealthy and stops routing to it. Until that threshold is hit, some requests can still land on a dying server, which is why health check tuning is a real production decision, not a default to ignore.
4. Layer Two: Auto Scaling Replaces What Dies, Automatically
An Auto Scaling Group (ASG) does two separate jobs that get confused for one: it scales the number of servers up and down with demand, and it replaces any server that fails a health check, regardless of demand. The second job is the one that matters for availability.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
BEFORE: AZ-B fails
+-----------+ +-----------+ +-----------+
| AZ-A | | AZ-B | | AZ-C |
| EC2 #1 | | EC2 #2 | | EC2 #3 |
| healthy | | X DEAD | | healthy |
+-----------+ +-----------+ +-----------+
ASG detects the failed health check on EC2 #2
|
v
AFTER: ASG launches a replacement in a surviving AZ
+-----------+ +-----------+ +-----------+
| AZ-A | | AZ-B | | AZ-C |
| EC2 #1 | | (down) | | EC2 #3 |
| healthy | | | | healthy |
+-----------+ +-----------+ | EC2 #4 |
| healthy |
+-----------+Define the ASG across at least two AZs with a minimum and desired capacity, and it keeps the desired count alive no matter which individual server or AZ disappears. This is the layer that turns "a server crashed" from an incident into a non-event.
5. Layer Three: The Database Has a Silent Twin
Stateless servers are the easy part. A database cannot just be re-launched elsewhere, because it holds data that has to match exactly. Amazon RDS Multi-AZ solves this with synchronous replication to a standby in a second AZ.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
NORMAL OPERATION
App --writes--> PRIMARY (AZ-A)
|
synchronous replication
|
v
STANDBY (AZ-B) <- not used for reads by default
AZ-A FAILS
App --writes--> X PRIMARY (AZ-A, dead)
RDS detects the failure, promotes the standby,
and flips the database's DNS endpoint to AZ-B
App --writes--> NEW PRIMARY (AZ-B, was standby)Two details matter here. First, replication to the Multi-AZ standby is synchronous: a write is not confirmed until the standby has it too, so failover does not lose committed data. Second, your application never changes its connection string. RDS re-points the same DNS endpoint at the new primary, and typical failover completes in roughly one to two minutes, though the exact figure depends on the engine and instance size.
Multi-AZ is for availability, not for scaling read traffic. For that, add Read Replicas, which are a separate feature layered on top.
6. Putting All Three Layers on One Diagram
Here is the complete picture: a request entering the system and surviving the loss of an entire AZ.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Users
|
v
+-------------------------+
| ALB (multi-AZ) |
+------------+-------------+
| |
+------+ +------+
v v
+----------------+ +----------------+
| AZ-A | | AZ-B |
| ASG instance | | ASG instance |
| RDS PRIMARY X FAILS | RDS STANDBY |
+----------------+ +--------+---------+
|
promoted to PRIMARY
DNS endpoint updated
v
App keeps writing,
same connection stringEvery arrow in that picture is a place a single point of failure could otherwise hide. That is the actual skill in designing for high availability: not knowing that Multi-AZ exists, but checking that every layer between the user and the data has its own twin.
7. What Multi-AZ Does Not Protect Against
- A bad deployment. If broken code or a broken migration ships to every AZ at once, redundancy does not help. That needs blue/green or canary deployments, a separate practice.
- A Region-wide event. Multi-AZ protects against losing one data center inside a Region. Surviving the loss of an entire Region needs multi-Region architecture, which is a much larger and more expensive undertaking.
- Human error at the data layer. A dropped table replicates to the standby just as faithfully as a saved one. Backups and point-in-time recovery are what undo that, not Multi-AZ.
8. A Quick Self-Check
- Is your load balancer, or equivalent entry point, spread across at least two AZs?
- Does your compute layer sit in an Auto Scaling Group with a minimum capacity greater than zero, across multiple AZs?
- Is your database running in Multi-AZ mode, not just a single instance with automated backups?
- Have you ever actually tested a failover, or only read that it exists?
If the answer to the last question is no, that is the next thing to do before calling any of this production-ready. AWS lets you trigger a manual RDS failover for exactly this kind of test.
Takeaway
High availability is not a checkbox, it is the absence of any single thing that, if it died tonight, would take your application with it. Multi-AZ gives you the building blocks. Whether they are actually wired together across all three layers is the part worth checking today.
Let's Support Each Other
If this article helped you, please like it and leave a comment. I will do my best to return the favour on your articles and posts, though replies may take a little time.
We are all working towards the 21-badge milestone, and genuine engagement makes that easier for everyone. Please read each other's articles, leave meaningful comments, and vote for good Wishlist ideas. I only have 5 Wishlist votes per day, so I cannot vote for everyone.
Learn, Build, Share, Connect, Earn, Get Certified. If that goal resonates with you, a vote means a lot. Thank you, and good luck with your own AWS goals.
#aws #cloudcomputing #highavailability #architecture #devops #AWSStudentRewards #AWSBuilderCenter #AWSCommunity #AWSCertification
Series: How to complete certification... (5 articles)
- …
- 5What Really Happens When an AWS Data Center Dies This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article