🚦 Day 12/30: Load Balancing — How Does One Application Handle Thousands of Users? ☁️
What happens when thousands of students open the exam portal at the same time? 🤔 Explore how load balancers distribute traffic across multiple EC2 servers, use health checks to avoid unhealthy instances, and help build highly available applications. If you were designing the college portal, would you use one powerful server or multiple servers behind a load balancer — and why? 💬
Series: Cloud Computing (24 articles)
- …
- 13🚦 Day 12/30: Load Balancing — How Does One Application Handle Thousands of Users? ☁️ This article
🚦 Day 12/30: Load Balancing — How Does One Application Handle Thousands of Users? ☁️
🎮 Mission 12: The Exam Registration Rush!
It’s 9:00 AM.
The college has opened examination registration. 📝
Thousands of students receive the notification at exactly the same time.
Rahul opens his laptop.
His friend opens the portal.
Then another student.
And another…
And suddenly…
10,000 students are trying to register at once! 😱
Our college portal is secure.
Our IAM permissions are carefully configured.
Our EC2 servers are running.
But there's a new problem…
1
2
3
4
5
6
7
8
9
👨🎓 👩🎓 👨🎓 👨🎓 👨🎓
👨🎓 👩🎓 👨🎓 👨🎓 👨🎓
👨🎓 👩🎓 👨🎓 👨🎓 👨🎓
|
v
🚨 10,000 Requests
|
v
??? SERVER ???If all those requests reach one EC2 server, that server could become overloaded.
So how do we distribute the traffic?
Welcome to Day 12.
Today we're introducing one of the most important components in scalable cloud architecture:
⚖️ Load Balancing
🏫 1. Imagine a College With One Registration Counter
Imagine our college has only one registration counter.
Thousands of students arrive.
1
2
3
4
5
6
7
8
9
10
👨🎓
👩🎓
👨🎓
👩🎓
👨🎓
👩🎓
👨🎓
|
v
🎫 One Registration CounterWhat happens?
The queue becomes huge.
Students wait.
The counter becomes overwhelmed.
Eventually, the whole process becomes slow.
Now imagine the college opens multiple counters.
1
2
3
4
5
6
7
8
9
10
11
👨🎓
👩🎓
👨🎓
👩🎓
|
v
🧑💼 Traffic Manager
/ | \
/ | \
v v v
🎫 Counter 1 🎫 Counter 2 🎫 Counter 3Now the students can be distributed across multiple counters.
That's the basic idea behind load balancing.
⚖️ 2. What Is a Load Balancer?
A load balancer receives incoming traffic and distributes requests across multiple application targets.
Instead of this:
1
2
3
4
5
6
7
Students
|
v
EC2 Server
|
v
🔥 OVERLOADEDWe can build this:
1
2
3
4
5
6
7
8
9
Students
|
v
Load Balancer
|
+--------+--------+
| | |
v v v
EC2-1 EC2-2 EC2-3Now the traffic can be spread across multiple application servers.
This gives our architecture a much better chance of handling traffic spikes.
🧠 3. The Load Balancer Doesn't Run Our Application
This is an important distinction.
A load balancer is not the application itself.
It doesn't normally contain our college registration code.
Instead, it acts as the traffic director.
Think of it like a receptionist.
A student says:
“I want to register for my examination.”
The receptionist decides which available registration counter should handle the request.
In cloud architecture:
1
2
3
4
5
6
7
8
9
10
Student
|
v
Load Balancer
|
+----> Application Server 1
|
+----> Application Server 2
|
+----> Application Server 3The application servers do the actual application work.
🚦 4. Meet Our AWS Application Load Balancer
For our web-based college portal, we can use an Application Load Balancer (ALB).
An ALB operates at the application layer and can route HTTP/HTTPS traffic to registered targets.
Our architecture now becomes:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
🌐 Internet
|
v
⚖️ Application
Load Balancer
|
+---------+---------+
| | |
v v v
EC2-1 EC2-2 EC2-3
| | |
+---------+---------+
|
v
DatabaseNow the registration traffic has somewhere to go besides one lonely server.
🩺 5. But What If One Server Is Broken?
Here's an interesting problem.
Imagine we have three application servers:
1
2
3
EC2-1 ✅
EC2-2 ❌
EC2-3 ✅EC2-2 has crashed.
Should the load balancer continue sending students there?
Obviously not!
That would be like sending students to a registration counter where nobody is working. 😄
This is where health checks become important.
🩺 6. Health Checks: “Are You Still Alive?”
A load balancer can use health checks to determine whether registered targets are healthy enough to receive traffic.
For example, the load balancer might check an application endpoint such as:
1
GET /healthThe application responds:
1
HTTP 200 OKThe load balancer knows:
“This server appears healthy.”
But if the server stops responding correctly:
1
2
3
EC2-1 ✅
EC2-2 ❌
EC2-3 ✅Traffic can be directed away from the unhealthy target according to the load balancer's configuration.
The flow becomes:
1
2
3
4
5
6
7
8
9
10
11
Load Balancer
|
+---------+---------+
| |
v v
EC2-1 EC2-3
HEALTHY HEALTHY
❌
EC2-2
UNHEALTHYThis is one reason load balancing works so well with highly available architectures.
🏗️ 7. Our College Portal Gets an Upgrade
Let's compare our architecture.
Before Load Balancing
1
2
3
4
5
6
7
Students
|
v
EC2
|
v
DatabaseOne application server is doing all the work.
Now look at the improved architecture:
After Load Balancing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Students
|
v
⚖️ Load Balancer
|
+------------+------------+
| | |
v v v
EC2-1 EC2-2 EC2-3
| | |
+------------+------------+
|
v
DatabaseMuch better!
But there's an even bigger improvement we can make.
🌍 8. Don't Put Everything in One Availability Zone
Remember Day 5?
We learned about AWS Regions and Availability Zones.
And on Day 6, we learned about high availability.
Let's use those lessons here.
Instead of placing all our EC2 servers in one Availability Zone:
1
2
3
4
5
Availability Zone 1
EC2-1
EC2-2
EC2-3We can distribute them across multiple Availability Zones.
1
2
3
4
5
6
7
8
9
10
⚖️ Load Balancer
|
+---------+---------+
| |
v v
Availability Zone 1 Availability Zone 2
EC2-1 EC2-3
EC2-2 EC2-4Now a problem affecting one Availability Zone doesn't automatically take down every application server.
This is where our previous lessons start connecting together.
🧩 9. Look How Our Architecture Has Evolved
Let's look at our journey.
Day 5
We asked:
Where does our application live?
We learned about Regions and Availability Zones.
Day 6
We asked:
What happens when something fails?
We introduced redundancy and high availability.
Day 7
We asked:
What actually runs our application?
We learned about virtual machines and EC2.
Day 8
We asked:
Where does our data live?
We explored S3, EBS, and EFS.
Day 9
We asked:
Where does application data live?
We introduced databases.
Day 10
We asked:
How does data travel through the cloud?
We explored VPCs, subnets, route tables, and security groups.
Day 11
We asked:
Who can access what?
We introduced IAM and least privilege.
And now…
Day 12
We're asking:
How can our application handle thousands of users?
The answer begins with:
⚖️ Load Balancing
🎯 10. A Load Balancer Gives Us More Than Traffic Distribution
Load balancing helps us build better architectures in several ways.
1️⃣ Traffic Distribution
Instead of sending every request to one server, traffic can be distributed across multiple healthy targets.
2️⃣ Better Availability
If a target becomes unhealthy, the load balancer can stop routing new requests to it according to its health-check configuration.
3️⃣ Scalability
We can run multiple application instances instead of depending on one server.
4️⃣ Central Entry Point
Users can access the application through a common endpoint while the backend contains multiple targets.
5️⃣ Application-Aware Routing
With an Application Load Balancer, routing can use HTTP/HTTPS information such as hostnames and URL paths.
For example:
1
2
3
4
5
6
7
8
9
example.com/exams
|
v
Exam Application
example.com/results
|
v
Results ApplicationThis becomes especially useful as our architecture grows.
🚨 11. The Exam Registration Rush Begins!
It's now 9:05 AM.
The registration portal opens.
Suddenly:
1
2
3
4
5
6
7
8
9
10
11
10 users
↓
100 users
↓
1,000 users
↓
5,000 users
↓
10,000 users
↓
🔥 TRAFFIC SPIKE!But our architecture looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
👨🎓 👩🎓 👨🎓
👩🎓 👨🎓 👩🎓
|
v
⚖️ Load Balancer
|
+------------+------------+
| | |
v v v
EC2-1 EC2-2 EC2-3
| | |
+------------+------------+
|
v
🗄️ DatabaseInstead of asking one server:
“Can you handle all 10,000 requests?”
We're giving the workload to a group of application servers.
But now comes the next question…
What happens when 10,000 users become 100,000 users?
🤔
Do we manually keep launching more EC2 instances?
That would be annoying.
And what happens when traffic drops at midnight?
Would we keep all those servers running?
That would waste resources and money.
And this leads us to our next major cloud concept…
🚀 12. The Secret Weapon: Auto Scaling
Imagine our college has a smart system that automatically opens more registration counters when the queue becomes huge.
1
2
3
4
5
6
7
8
9
Low Traffic
Students
|
v
Load Balancer
|
+----> EC2-1
+----> EC2-2Traffic increases…
1
2
3
4
5
6
7
8
9
10
11
High Traffic
Students
|
v
Load Balancer
|
+----> EC2-1
+----> EC2-2
+----> EC2-3
+----> EC2-4Traffic decreases…
1
2
3
4
5
6
7
8
9
Low Traffic Again
Students
|
v
Load Balancer
|
+----> EC2-1
+----> EC2-2This idea leads directly to Auto Scaling.
But we'll save that mission for the next stage of our journey.
🧠 13. Quick Challenge: Choose the Right Architecture
Imagine you are designing the college examination portal.
Which architecture would you choose?
Option A
1
2
3
4
5
6
7
Students
|
v
One EC2 Server
|
v
DatabaseOption B
1
2
3
4
5
6
7
8
9
10
11
Students
|
v
Load Balancer
|
+----> EC2-1
+----> EC2-2
+----> EC2-3
|
v
DatabaseOption C
1
2
3
4
5
6
7
8
9
10
Students
|
v
One EC2 Server
|
v
Load Balancer
|
v
DatabaseThink carefully…
Which one makes the most sense when many users need to access the application?
✅ Answer
Option B
Because the load balancer can distribute incoming application traffic across multiple application targets.
🧪 14. Architecture Challenge
Now imagine this situation:
1
2
3
4
5
6
7
8
9
10
Internet
|
v
Load Balancer
|
+--------+--------+
| | |
v v v
EC2-1 EC2-2 EC2-3
✅ ❌ ✅EC2-2 fails its health check.
Question:
Should the load balancer continue sending new requests to EC2-2?
Answer: Generally, no.
The load balancer can stop routing new requests to an unhealthy target based on its health-check configuration.
Now imagine EC2-2 becomes healthy again.
1
2
EC2-1 EC2-2 EC2-3
✅ ✅ ✅It can become eligible to receive traffic again.
That's the power of combining:
Load Balancing + Health Checks + Multiple Application Instances
🏆 Mission Complete!
Our college examination portal has survived another major challenge.
We started with:
1
2
3
4
Students
|
v
One Application ServerAnd evolved into:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Students
|
v
⚖️ Load Balancer
|
+--------------+--------------+
| | |
v v v
EC2-1 EC2-2 EC2-3
| | |
+--------------+--------------+
|
v
DatabaseOur architecture is becoming more resilient and scalable.
But there is still a problem.
What happens when traffic changes automatically?
At 2 AM, perhaps only 20 students are using the portal.
At 9 AM, thousands arrive.
At 10 AM, the traffic doubles.
At 2 PM, it drops again.
Should we manually add and remove servers every time?
Absolutely not. 😄
🚀 Coming Up on Day 13/30
📈 Auto Scaling — How Does AWS Automatically Add and Remove Servers?
Tomorrow, our college portal faces its biggest traffic spike yet.
Instead of manually launching EC2 instances, we'll teach our architecture to scale automatically based on demand.
We'll discover:
- 📈 What is Auto Scaling?
- ⚙️ How does an Auto Scaling Group work?
- 🖥️ How are EC2 instances added and removed?
- ❤️ How do health checks help replace unhealthy instances?
- ⚖️ How does Auto Scaling work together with a Load Balancer?
- 💰 Why scaling down can also save money?
Our architecture is getting smarter.
And the cloud journey continues…
☁️ Build it. Secure it. Scale it.
#CloudComputing #AWS #CloudArchitecture #LoadBalancing #AWSALB #AmazonEC2 #30DaysOfCloud
Series: Cloud Computing (24 articles)
- …
- 13🚦 Day 12/30: Load Balancing — How Does One Application Handle Thousands of Users? ☁️ This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article