AWS Builder Center

How Amazon EC2 Actually Works Under the Hood

Learn how Amazon EC2 works under the hood, from virtualization and Nitro to CPU, memory, storage, networking, and security.

I learn
When I first started learning about Amazon EC2, I thought I understood it.
I knew that EC2 lets us launch virtual servers in the cloud. We choose an instance type, select an AMI, configure networking, and AWS gives us a machine that we can connect to.
But then I started wondering:
What is actually happening behind the scenes when I click "Launch instance"?
Where does the virtual machine come from?
How does AWS make one physical server run many EC2 instances?
How does my instance get CPU, RAM, storage, and networking?
And perhaps the most interesting question:
How does AWS make a virtual machine feel like a real computer?
So I started digging deeper into how EC2 works under the hood.
Here is what I understood.

First, What Is EC2 Really?

At a high level, Amazon EC2 is a service that provides virtualized compute capacity.
When we launch an EC2 instance, we aren't normally receiving an entire physical server dedicated exclusively to us.
Instead, AWS operates enormous physical data centers containing physical servers.
Those servers have:
  • CPUs
  • Memory
  • Storage
  • Network interfaces
  • Network connectivity
  • Power and cooling
  • Hardware monitoring
AWS uses virtualization and other infrastructure technologies to divide the resources of physical machines into isolated compute environments.
So the simplified picture looks something like this:
Physical AWS Server
↓
Virtualization / AWS infrastructure
↓
EC2 Instance
↓
Operating System
↓
Applications
That simple diagram helped me understand EC2 much better.

What Happens When I Launch an EC2 Instance?

Let's imagine that I open the AWS Console and click:
Launch instance
It looks like one simple action.
But underneath, AWS has to make several decisions and perform many operations.
Conceptually, the process looks something like:
1. Choose an AWS Region
↓
2. Choose an Availability Zone
↓
3. Select an AMI
↓
4. Select an instance type
↓
5. Configure networking
↓
6. Allocate compute resources
↓
7. Attach storage
↓
8. Configure security
↓
9. Start the virtual machine
↓
10. Boot the operating system
The whole process can happen in a short amount of time.
But there is a lot going on underneath.



Where Does My EC2 Instance Actually Run?

This was one of the first things I wanted to understand.
If I launch an EC2 instance, where is it physically?
The answer is:
On AWS physical infrastructure inside the Availability Zone I selected.
AWS operates physical data centers around the world.
These contain large numbers of physical servers.
Those servers are often called host systems in the context of virtualization.
A physical host might have a large number of CPU cores and a large amount of memory.
Instead of giving all of those resources to a single customer, AWS can use virtualization to run multiple isolated instances on the underlying hardware.
Conceptually:
Physical Host
│
├── EC2 Instance A
│ ├── vCPU
│ ├── Memory
│ └── Storage
│
├── EC2 Instance B
│ ├── vCPU
│ ├── Memory
│ └── Storage
│
└── EC2 Instance C
├── vCPU
├── Memory
└── Storage
The instances are isolated from one another even though they ultimately depend on physical infrastructure.
This is one of the fundamental ideas behind cloud computing.



The Hypervisor

This is where things become more interesting.
How can multiple operating systems run on the same physical machine without interfering with one another?
One important part of the answer is virtualization.
A virtualization layer, commonly called a hypervisor, allows physical hardware resources to be divided into virtual machines.
You can think of the hypervisor as a layer between the physical hardware and the virtual machines.
┌─────────────────────────────┐
│ EC2 Instance A │
│ Linux / Windows │
├─────────────────────────────┤
│ EC2 Instance B │
│ Linux / Windows │
├─────────────────────────────┤
│ EC2 Instance C │
│ Linux / Windows │
└─────────────────────────────┘
│
Virtualization
│
┌─────────────────────────────┐
│ Physical Server │
│ CPU | RAM | Network | ... │
└─────────────────────────────┘
AWS has evolved its virtualization architecture significantly over time.
Modern Nitro-based EC2 instances use the AWS Nitro System, which moves many traditional virtualization and infrastructure functions into dedicated hardware and firmware.
That is one reason EC2 today can provide strong performance while maintaining isolation between customers.



What Is the AWS Nitro System?

This was probably the most interesting part of my research.
When I first heard about Nitro, I assumed it was simply another AWS service.
It isn't.
The AWS Nitro System is a collection of AWS-designed hardware and software components used to build EC2 infrastructure.
AWS describes Nitro as consisting of components such as:
  • Nitro Cards
  • Nitro Security Chip
  • Nitro Hypervisor
The goal is to offload infrastructure functions from the main server CPUs and provide strong isolation and performance.
A simplified architecture looks like this:
EC2 Instance
│
┌──────────┴──────────┐
│ │
vCPU Memory
│ │
└──────────┬──────────┘
│
Nitro Hypervisor
│
┌──────────┼──────────┐
│ │ │
Network Storage Security
│ │ │
└──────────┼──────────┘
│
Physical Hardware
The actual architecture is more complicated, but this mental model helped me understand why Nitro matters.



Where Does the CPU Come From?

When we choose an EC2 instance such as a particular instance size, we're essentially requesting a certain amount of compute capacity.
For example, an instance type might provide a certain number of vCPUs and a specific amount of memory.
But a vCPU isn't a separate physical CPU sitting inside the cloud.
It represents virtualized CPU capacity assigned to the instance.
The physical host contains actual CPU cores.
AWS's virtualization infrastructure maps the virtual CPU resources presented to the guest operating system onto physical CPU resources.
So when my application executes:
x = 10 + 20
the application thinks it is running on a CPU inside its own computer.
But underneath:
Application
↓
Operating System
↓
Virtual CPU
↓
AWS virtualization infrastructure
↓
Physical CPU
The important thing is that the operating system doesn't need to know which physical CPU core is ultimately executing the instruction.
It sees the virtual hardware presented to it.



What About RAM?

Memory works in a similar conceptual way.
Suppose I launch an instance with several gigabytes of RAM.
The operating system sees something like:
RAM
├── Kernel
├── Applications
├── Buffers
└── Cache
But physically, that memory is backed by memory resources on AWS infrastructure.
The virtualization layer ensures that my instance can access the memory assigned to it while preventing it from accessing another customer's memory.
This isolation is extremely important.
Imagine if one EC2 instance could simply read another customer's memory.
That would be a massive security problem.
AWS's virtualization architecture is designed specifically to prevent that kind of cross-instance access.



What Happens When the Operating System Boots?

This part helped me understand EC2 as a computer rather than just an AWS service.
When an EC2 instance starts, it needs an operating system.
That's where an AMI, or Amazon Machine Image, comes in.
An AMI contains the information required to launch an instance, including the operating system and configuration needed to boot it.
For example, I might choose an Amazon Linux AMI.
Conceptually:
AMI
│
├── Boot configuration
├── Operating system
├── Filesystem
└── Software / configuration
AWS uses that image as the basis for creating the instance's boot environment.
The operating system then starts just as an operating system would on a physical computer.
It initializes:
  • CPU
  • Memory
  • Devices
  • Network interfaces
  • Storage
  • Kernel
  • System services
Eventually, I get a running Linux or Windows environment.
From my perspective:
ssh user@my-server
and I'm inside a computer.
But that computer is virtualized.



Where Does the Disk Come From?

This was another thing I initially misunderstood.
I thought:
"If EC2 is a virtual machine, where is its hard disk?"
The answer depends on the storage configuration.
One common option is Amazon EBS (Elastic Block Store).
EBS provides persistent block storage that can be attached to EC2 instances.
Conceptually:
EC2 Instance
│
│ block storage
▼
EBS Volume
│
▼
AWS Storage Infrastructure
The operating system sees the EBS volume as a block device.
For example, Linux might show something like:
/dev/nvme0n1
The application doesn't need to know that the underlying storage infrastructure is distributed across AWS hardware.
It simply reads and writes blocks.
That abstraction is extremely powerful.



What Happens to My Data When I Write a File?

Suppose I run:
echo "Hello AWS" > hello.txt
It looks simple.
But several layers are involved.
Conceptually:
Application
↓
Linux filesystem
↓
Block device
↓
Virtual storage interface
↓
EBS infrastructure
↓
Physical storage
The application doesn't directly control the physical disk.
Instead, it communicates with the operating system.
The operating system communicates with the virtual block device.
AWS infrastructure handles the underlying storage.
This separation allows AWS to provide features such as snapshots, volume resizing, encryption, and high availability without requiring applications to understand the physical storage system.



How Does Networking Work?

This is where EC2 becomes even more interesting.
When I launch an instance inside a VPC, it receives network connectivity through a virtual network interface called an Elastic Network Interface (ENI).
The operating system sees a network interface.
For example:
eth0
It might have:
  • A private IP address
  • A subnet
  • Security group associations
  • MAC address information
But there isn't necessarily a physical Ethernet cable running directly from my virtual machine to a switch.
Instead, AWS provides virtual networking backed by physical networking infrastructure.
Conceptually:
EC2
│
▼
Virtual Network Interface
│
▼
VPC
│
├── Subnet
├── Route Table
├── Security Group
└── Network infrastructure
│
▼
AWS Network
That is how an EC2 instance can communicate with other resources while still being logically isolated from other customers.



What Is a Security Group Doing?

When I create an EC2 instance, AWS asks me to configure a security group.
At first, I thought of it as a firewall attached directly to the operating system.
But that's not quite right.
A security group is an AWS-level virtual firewall associated with network interfaces.
For example, I might allow:
TCP 22 → SSH
TCP 80 → HTTP
TCP 443 → HTTPS
The operating system may also have its own firewall, such as  iptables  or  nftables .
So there can be multiple layers of network security:
Internet
↓
AWS networking
↓
Security Group
↓
EC2 network interface
↓
Operating system firewall
↓
Application
This layered approach is one reason cloud security requires understanding both AWS networking and the operating system.



How Does AWS Give My Instance an IP Address?

When an instance is launched inside a subnet, AWS assigns it network configuration according to the VPC and subnet configuration.
For example:
VPC
10.0.0.0/16

↓

Subnet
10.0.1.0/24

↓

EC2
10.0.1.25
The private IP address belongs to the instance's network interface.
If I also associate a public IPv4 address, AWS provides the necessary networking to make that instance reachable from the internet, subject to routing and security rules.
So the public IP isn't simply a physical address assigned to a server.
It is part of AWS's virtual networking system.



What Happens When I Connect Using SSH?

Let's say my EC2 instance has a public IP.
I run:
ssh -i my-key.pem ec2-user@203.0.113.10
What happens?
A simplified path looks like:
My Computer
│
▼
Internet
│
▼
AWS Network
│
▼
VPC
│
▼
Security Group
│
▼
EC2 Network Interface
│
▼
Operating System
│
▼
SSH Server
If everything is configured correctly, the SSH daemon accepts the connection.
This is why an EC2 instance can feel exactly like a traditional remote server.
The difference is that the physical infrastructure underneath it is managed by AWS.



But Where Is the Physical Server?

This question gets interesting because AWS intentionally abstracts away much of the physical infrastructure.
When I launch an EC2 instance, I usually don't choose:
"Put my server in Rack 17, Server 42."
Instead, I choose things such as:
  • Region
  • Availability Zone
  • Instance type
  • AMI
  • VPC
  • Subnet
  • Storage
  • Security configuration
AWS's infrastructure determines where the workload can run within those constraints.
For standard shared-tenancy EC2 instances, AWS manages the physical host placement.
For certain specialized options, customers can get more control over physical placement or dedicated hardware.
This is another important cloud concept:
You specify what infrastructure characteristics you need rather than manually managing every physical machine.



What Happens When I Stop an Instance?

This is something that confused me at first.
When I stop an EBS-backed EC2 instance, the virtual machine is no longer running.
The compute resources are released.
But the EBS-backed storage can remain.
So conceptually:
Running:

EC2
↓
CPU + RAM
↓
EBS

Stopped:

EC2 compute → released
EBS volume → remains
When I start the instance again, AWS allocates compute resources and attaches the persistent storage.
The operating system can then boot again.
This is one reason compute and storage are treated as separate resources in the cloud.



What Happens When I Terminate an Instance?

Termination is different.
When an instance is terminated, AWS removes the instance's compute environment.
Depending on the storage configuration and deletion settings, attached EBS volumes may also be deleted.
That means I should never assume:
"My EC2 instance is my data."
A better mental model is:
EC2 = compute
EBS = persistent block storage
That distinction becomes extremely important when designing production systems.



What About Auto Scaling?

Now imagine I have one EC2 instance running a web application.
Suddenly, traffic increases.
One server isn't enough.
Instead of manually launching servers every time, I can use an Auto Scaling Group.
Conceptually:
Load Balancer
│
┌───────────┼───────────┐
↓ ↓ ↓
EC2 EC2 EC2
Instance 1 Instance 2 Instance 3
If traffic increases, additional instances can be launched.
If traffic decreases, instances can be removed.
This is one of the fundamental advantages of cloud computing.
The infrastructure can respond to demand.



But What If a Physical Server Fails?

This is where AWS's infrastructure design becomes important.
Physical hardware can fail.
CPUs can fail.
Memory can fail.
Network components can fail.
Power equipment can fail.
AWS therefore designs its infrastructure around failure rather than assuming failure won't happen.
At the EC2 level, customers can use multiple Availability Zones.
For example:
Region
│
├── Availability Zone A
│ ├── EC2
│ └── EC2
│
├── Availability Zone B
│ ├── EC2
│ └── EC2
│
└── Availability Zone C
├── EC2
└── EC2
If my application is designed correctly, losing one Availability Zone doesn't necessarily mean losing the entire application.
This is a major difference between simply running a server and designing a resilient system.



EC2 Is More Than a Virtual Machine

This was probably my biggest takeaway.
When people explain EC2, they often say:
"EC2 is a virtual machine."
That's useful for beginners, but it doesn't tell the whole story.
EC2 is actually a combination of several layers:
Application
│
▼
Operating System
│
▼
Virtual Hardware
│
┌──────────────┼──────────────┐
▼ ▼ ▼
vCPU Memory Network
│ │ │
└──────────────┼──────────────┘
▼
Nitro System
│
┌──────────────┼──────────────┐
▼ ▼ ▼
Physical Storage Networking
Hardware Infrastructure Infrastructure
The exact implementation is obviously much more complicated.
But this mental model helped me understand what AWS is actually providing.



The Interesting Part: AWS Hides the Complexity

Imagine if every time I launched an EC2 instance, I had to personally worry about:
  • Which physical server should I use?
  • Which rack?
  • Which CPU?
  • Which memory modules?
  • Which network switch?
  • Which physical disk?
  • How should the server connect to the internet?
  • How should I isolate it from another customer's server?
  • How should hardware failures be handled?
Cloud computing removes most of those concerns.
Instead, I can say:
I need:
4 vCPUs
16 GB RAM
100 GB storage
A Linux operating system
A private network
Internet access
AWS turns that request into a running compute environment.
That abstraction is the real power of EC2.



The Full Picture

After learning all these pieces, this is the mental model that finally made EC2 click for me:
AWS Region
│
┌──────────┴──────────┐
│ │
AZ-A AZ-B
│ │
Physical Hosts Physical Hosts
│ │
┌─────┴─────┐ ┌─────┴─────┐
│ │ │ │
EC2 EC2 EC2 EC2
│ │ │ │
vCPU vCPU vCPU vCPU
RAM RAM RAM RAM
│ │ │ │
└───────────┴─────────┴───────────┘
│
VPC
│
┌─────────┼─────────┐
│ │ │
EBS ENI Security
Storage Network Groups
Again, this isn't a literal diagram of every AWS internal component.
It's a way to understand the relationship between the pieces.



What Finally Clicked for Me

Before learning about EC2 internals, I thought:
"AWS gives me a virtual computer."
Now I think about it differently.
AWS operates massive physical infrastructure.
On top of that infrastructure, AWS provides virtualization, networking, storage, security, and management systems.
Those systems allow AWS to expose compute resources as EC2 instances.
So when I launch an EC2 instance, I'm not creating a physical server.
I'm requesting a managed slice of AWS's enormous computing infrastructure, presented to me as a virtual computer.
And that's why cloud computing is so powerful.
I don't need to own the data center.
I don't need to buy the servers.
I don't need to install physical networking equipment.
I don't need to manually replace failed hardware.
I can simply request the resources I need through an API or console and start building.

Final Thoughts

Understanding EC2 at this level changed how I think about AWS.
EC2 isn't magic.
Underneath the AWS console are:
physical servers
CPUs
memory
storage systems
networking hardware
virtualization
security mechanisms
and a huge amount of automation.
AWS takes all of that complexity and exposes a much simpler interface:
"Here is your virtual server. Build something."
And I think that's one of the most important ideas to understand when learning cloud computing.
The cloud doesn't eliminate infrastructure.
It abstracts infrastructure.
Once that clicked for me, EC2 stopped feeling like a mysterious AWS service and started feeling like what it really is:
a carefully engineered abstraction over massive amounts of physical computing infrastructure.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article