AWS Builder Center

Builder Center

The Capital One Breach Taught Me Why My EC2 Instance Has No AWS Keys

The Capital One Breach Taught Me Why My EC2 Instance Has No AWS Keys

How did one HTTP request leak data on 100M+ Capital One customers? A look at IAM roles, IMDS, and why IMDSv2's token flow blocks SSRF, with Terraform you can copy

Gustavo Benites

System and process enhancer.

The Capital One Breach Taught Me Why My EC2 Instance Has No AWS Keys

In 2019, data on more than 100 million Capital One customers left their S3 buckets. The attacker didn't crack a password or steal a laptop. They tricked a server into making one HTTP request to an address that only the server itself can reach. That address answered with valid AWS credentials.
I came across this story while studying IAM for the AWS Certified Cloud Practitioner exam. It changed how I think about credentials on EC2, and it's the reason my Terraform lab has a line that says http_tokens = "required".
TL;DR* A 24/7 EC2 instance doesn't need permanent AWS keys. An IAM role plus the Instance Metadata Service (IMDS) gives it temporary, auto-rotating credentials.
  • IMDS can only be reached from inside the instance, but that doesn't protect it from SSRF. That's how the Capital One attacker got in.
  • Require IMDSv2 and give the role least privilege. In Terraform, the first part takes one line.

The question that started it

If an EC2 instance runs 24/7 and needs to read from S3, doesn't it need a permanent access key stored somewhere?
My first guess was "yes". A server that never sleeps should need a credential that never expires. That guess is wrong.

How AWS API calls are authenticated

Every AWS API call, including every S3 GetObject, is an HTTPS request signed with Signature Version 4 (SigV4). The signature comes from an access key ID, a secret access key and, if the credentials are temporary, a session token. The secret key itself never goes over the network. Only a signature derived from it does.
So the real question is "where does my code get a key to sign with, and how long does that key last?"

The answer: an IAM role and IMDS

Attach an IAM role to the instance through an instance profile. Every EC2 instance can reach the Instance Metadata Service (IMDS) at 169.254.169.254, a link-local address that only that instance can reach. When a role is attached, IMDS hands out temporary credentials that rotate automatically, and the AWS SDKs and CLI find them on their own:
1
2
3
4
5
6
import boto3

# No keys in code, no env vars, no ~/.aws/credentials file.
# The SDK gets the role's temporary credentials from IMDS.
s3 = boto3.client("s3")
s3.list_objects_v2(Bucket="my-bucket")
"Always-on" does not mean "needs permanent credentials." The instance runs forever, but no single credential on it lasts more than a few hours. Nothing permanent sits on disk to leak or get committed to Git.
On your laptop there's no IMDS. The equivalent is IAM Identity Center (SSO): run aws configure sso and then aws sso login to get short-lived credentials instead of long-term access keys.

Only reachable from inside isn't the same as safe

It's tempting to treat 169.254.169.254 as safe because nobody outside can reach it. But that does nothing against an attack that comes from inside the instance.
If an app on the box has a Server-Side Request Forgery (SSRF) flaw, an attacker can make it request any URL they choose, including the metadata endpoint. IMDS can't tell "the app is asking" from "an attacker is controlling the app". It just answers.

What happened at Capital One

  1. A misconfigured web application firewall running on EC2 had an SSRF flaw.
  2. The attacker used it to query IMDSv1 and got the instance role's temporary credentials.
  3. Those credentials had broad S3 permissions, so the attacker downloaded customer data at scale.
Temporary credentials didn't save them. They were temporary, but they were valid and had far too much permission. Two lessons came out of it: protect the metadata endpoint, and scope the role.

IMDSv2: how a token blocks SSRF

AWS's answer is IMDSv2. Instead of one simple GET, a caller now has to send a PUT to get a session token, then include that token in every GET:
1
2
3
4
5
6
7
# 1. Get a session token (PUT, with a TTL header)
TOKEN=$(curl -sX PUT "http://169.254.169.254/latest/api/token" \
-H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

# 2. Use it to read metadata, e.g. the attached role's name
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/iam/security-credentials/
When IMDSv2 is required, a plain GET with no token gets 401 Unauthorized. This blocks most SSRF attacks for three reasons:
  • Most SSRF bugs can only force a simple GET. They can't send a PUT with custom headers and then reuse the response in a second request.
  • Token requests that carry an X-Forwarded-For header are rejected, so an open reverse proxy on the box can't relay them.
  • Hop limit: the token response is sent with a low IP TTL. With http_put_response_hop_limit = 1, the token can't travel past the instance itself.

The Terraform part

In my lab, a t4g.nano (Graviton/ARM) instance running Amazon Linux 2023, enforcing IMDSv2 is one setting inside metadata_options:
1
2
3
4
5
6
7
8
9
10
11
resource "aws_instance" "lab" {
ami = data.aws_ami.al2023_arm.id
instance_type = "t4g.nano"
vpc_security_group_ids = [aws_security_group.ssh.id]

metadata_options {
http_endpoint = "enabled"
http_tokens = "required" # the line that matters: "optional" still allows IMDSv1
http_put_response_hop_limit = 1
}
}
To give the instance S3 access the right way, attach a role through an instance profile. Never put keys in user_data:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
resource "aws_iam_role" "ec2_s3_read" {
name = "ec2-lab-s3-read"

# Trust policy: only the EC2 service can assume this role.
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "ec2.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
}

# Fine for a lab. In real life, scope it to the one bucket the app needs.
resource "aws_iam_role_policy_attachment" "s3_read" {
role = aws_iam_role.ec2_s3_read.name
policy_arn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}

resource "aws_iam_instance_profile" "lab" {
name = "ec2-lab-profile"
role = aws_iam_role.ec2_s3_read.name
}

# ...and in aws_instance.lab:
# iam_instance_profile = aws_iam_instance_profile.lab.name

Takeaways

  • Every AWS API call is SigV4-signed, and the secret key never goes over the network.
  • On EC2, use an IAM role and IMDS: no permanent credentials, even for a 24/7 server.
  • On a laptop, prefer IAM Identity Center (SSO) over long-term access keys.
  • "Only reachable from inside the instance" isn't a security boundary against SSRF.
  • Require IMDSv2 (http_tokens = "required") and keep the hop limit low.
  • Temporary credentials still need least privilege. The problem at Capital One was what the credentials were allowed to do.
For Cloud Practitioner candidates: this topic covers two exam domains. Security and Compliance covers IAM roles, least privilege, and the shared responsibility model: AWS runs IMDS, but enforcing IMDSv2 and scoping the role are your job. Cloud Technology and Services covers EC2 and S3. Learning the story behind the rule made the definitions much easier to remember.
Over to you: do you enforce IMDSv2 across your accounts, and how? Through Terraform, an SCP, or the EC2 account-level default? I'd like to hear what works in real environments.
# amazon-ec2# iam# security# terraform# training-and-certification
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article