Behind the Scenes: EC2 Compute Capacity
A look behind the infrastructure that powers Amazon EC2, covering physical hosts, virtualization, the Nitro System, instance types, and compute capacity across Availability Zones.
Series: Behind the Scenes (3 articles)
- 3Behind the Scenes: EC2 Compute Capacity This article
EC2 Compute Capacity
When an EC2 instance is launched, it is easy to think of it simply as creating a virtual machine. In reality, AWS needs to provide physical compute resources somewhere in its infrastructure before that virtual machine can run.
Understanding this relationship between physical infrastructure and virtual compute helps explain how EC2 achieves scalability and why concepts such as Availability Zones and instance types matter.
AWS Infrastructure Hierarchy
AWS infrastructure is organized into Regions, which contain multiple Availability Zones (AZs).
An Availability Zone consists of one or more discrete data centers with independent infrastructure.
When an EC2 instance is launched into a particular Availability Zone, AWS needs to provide the required compute capacity within that AZ.
This means that an EC2 instance is ultimately backed by physical infrastructure, even though it appears to the user as a virtual server.
A simplified hierarchy looks like:
Region → Availability Zone → Physical Infrastructure → EC2 Instance
Physical Hosts and Shared Infrastructure
EC2 instances run on physical servers managed by AWS.
These physical machines provide resources such as CPU, memory, networking, and sometimes local storage.
Multiple EC2 instances can run on the same underlying physical infrastructure while remaining isolated from each other.
This is possible through virtualization.
The physical server therefore becomes a shared resource from which AWS can provide multiple virtual compute environments.
Virtualization and Resource Allocation
When an EC2 instance is launched, AWS needs to allocate the resources required by its selected instance type.
For example, an instance type may specify a certain number of vCPUs and a specific amount of memory.
The physical infrastructure provides the actual CPU and memory, while the EC2 instance receives a virtualized representation of these resources.
From inside the operating system, the instance behaves like an independent machine.
The underlying physical server is abstracted away.
This abstraction is one of the fundamental ideas behind cloud computing:
Physical resources → Virtual resources → Cloud services
The AWS Nitro System
Modern EC2 instances are built on the AWS Nitro System, which provides the underlying architecture for virtualization and hardware-level functions.
Instead of placing all virtualization and infrastructure functions directly on the main CPU, Nitro uses dedicated hardware components for tasks such as networking, storage, and virtualization management.
This allows the compute resources to be focused more directly on the workloads running inside the EC2 instances.
It also provides strong isolation between instances sharing the same physical infrastructure.
The important point is that users do not need to manage the physical host or the virtualization layer. AWS manages these components as part of the EC2 service.
Instance Types and Capacity
An EC2 instance type is essentially a predefined combination of compute resources.
Different instance families are designed around different resource requirements.
For example:
- Compute Optimized → Higher CPU capability
- Memory Optimized → Larger memory capacity
- General Purpose → Balanced resources
- Storage Optimized → High local storage performance
- Accelerated Computing → GPUs or other accelerators
When a specific instance type is requested, AWS needs to find available capacity capable of providing that configuration.
This is why an instance type is not simply a software setting. It represents a particular allocation of underlying infrastructure.
EC2 Capacity Availability
Sometimes an EC2 launch can fail with an InsufficientInstanceCapacity error.
This does not necessarily mean that AWS has no servers available globally.
It generally means that the requested capacity is temporarily unavailable for the particular combination of instance type and Availability Zone.
For example, there may be sufficient capacity for one instance type in an AZ but not for another.
This is one reason why applications requiring high availability are commonly distributed across multiple Availability Zones.
Capacity Distribution Across Availability Zones
Consider an application that requires four EC2 instances.
If all four are placed in a single Availability Zone, the application depends heavily on the capacity and availability of that AZ.
A more resilient architecture could distribute the instances:
Availability Zone A → 2 instances
Availability Zone B → 2 instances
Availability Zone B → 2 instances
This provides both higher availability and more flexibility when capacity needs to be allocated.
The EC2 Capacity Flow
The process can be simplified as:
EC2 Request
↓
Instance Type + AZ Requirements
↓
Available Physical Capacity
↓
Virtual Compute Resources Allocated
↓
EC2 Instance Created
↓
Operating System Starts
↓
Instance Type + AZ Requirements
↓
Available Physical Capacity
↓
Virtual Compute Resources Allocated
↓
EC2 Instance Created
↓
Operating System Starts
The actual scheduling and infrastructure management behind this process are much more complex, but the basic concept is important:
An EC2 instance is a virtual representation of physical resources managed by AWS.
Series: Behind the Scenes (3 articles)
- 3Behind the Scenes: EC2 Compute Capacity This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article