AWS Builder Center
My Pod Was Running, But AWS Still Said AccessDenied: Understanding EKS Pod Identity

My Pod Was Running, But AWS Still Said AccessDenied: Understanding EKS Pod Identity

A practical deep dive into one of the most confusing EKS problems: a healthy Pod that can communicate with Kubernetes but cannot access AWS services. This article explains EKS Pod Identity, ServiceAccounts, IAM trust policies, debugging commands, common failure points, and a step-by-step troubleshooting workflow.

I'm a student, learning and growing day by day.

My Pod Was Running, But AWS Still Said AccessDenied: Understanding EKS Pod Identity

One of the most confusing moments when learning Amazon EKS is seeing something like this:
1
2
3
4
5
6
Pod: Running
Service: Running
Deployment: Ready

Application:
AccessDeniedException
At first, it feels strange.
If the Pod is running correctly, why can't the application access Amazon S3, DynamoDB, Secrets Manager, or another AWS service?
The answer is that Kubernetes and AWS IAM solve different identity and authorization problems.
That distinction became much clearer when I started looking at the complete request path:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Kubernetes Pod
|
v
ServiceAccount
|
v
EKS Pod Identity
|
v
IAM Role
|
v
IAM Policy
|
v
AWS Service
Understanding this flow makes a large class of EKS AccessDenied problems much easier to reason about.

1. The Problem: A Healthy Pod Can Still Be Unauthorized

Imagine an application running in EKS that needs to read objects from an S3 bucket.
The Kubernetes resources might all look healthy:
1
kubectl get pods
1
2
NAME READY STATUS
backend-7d8c6f9d8f-x2k7p 1/1 Running
The Deployment is healthy:
1
kubectl get deployment
1
2
NAME READY UP-TO-DATE AVAILABLE
backend 1/1 1 1
But the application logs show:
1
AccessDenied
This is where an important question appears:
Who is the application when it talks to AWS?
The answer is not simply:
"The Pod."
The application ultimately needs AWS credentials associated with an IAM role that has permission to perform the requested AWS API operation.

2. Kubernetes Identity and AWS Identity Are Different

This is the mental model that helped me understand EKS authentication.
Kubernetes has concepts such as:
  • Users
  • ServiceAccounts
  • Roles
  • RoleBindings
  • ClusterRoles
  • ClusterRoleBindings
AWS has:
  • IAM users
  • IAM roles
  • IAM policies
  • STS
  • Resource policies
They are related in an EKS environment, but they are not the same authorization system.
For example:
1
2
3
4
5
6
7
Kubernetes authorization

Pod
|
+--> ServiceAccount
|
+--> Kubernetes RBAC
is different from:
1
2
3
4
5
6
7
8
9
AWS authorization

Application
|
+--> IAM Role
|
+--> IAM Policy
|
+--> S3 / DynamoDB / Secrets Manager / ...
A Pod can therefore be perfectly healthy from Kubernetes' perspective while its application is unauthorized to call an AWS API.

3. Enter EKS Pod Identity

Amazon EKS provides EKS Pod Identity to give workloads running in Pods permissions to access AWS services.
Instead of placing long-lived AWS credentials inside the container, an IAM role can be associated with a Kubernetes ServiceAccount.
AWS describes this as a least-privilege mechanism because permissions can be scoped to the ServiceAccount used by the workload. ([AWS Documentation][1])
The simplified architecture looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Amazon EKS
|
v
+----------------+
| ServiceAccount |
+----------------+
|
v
+----------------+
| EKS Pod |
| Identity |
+----------------+
|
v
+----------------+
| IAM Role |
+----------------+
|
v
+----------------+
| IAM Policy |
+----------------+
|
v
Amazon S3
The application can then use AWS SDKs through the normal credential provider chain.

4. A Simple Example

Suppose I have an application called:
1
backend
and I want it to read from an S3 bucket.
First, I create a ServiceAccount:
1
2
3
4
5
apiVersion: v1
kind: ServiceAccount
metadata:
name: backend-sa
namespace: default
Then my Deployment uses it:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 2

selector:
matchLabels:
app: backend

template:
metadata:
labels:
app: backend

spec:
serviceAccountName: backend-sa

containers:
- name: backend
image: my-backend:latest
The important line is:
1
serviceAccountName: backend-sa
Without this, the Pod may run using a different ServiceAccount than the one I intended.

5. The IAM Role Is the Actual AWS Permission Boundary

The ServiceAccount alone does not magically give the application S3 access.
The IAM role needs an appropriate permissions policy.
For example, conceptually:
1
2
3
4
5
6
7
8
9
10
11
12
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::my-application-bucket/*"
}
]
}
This is where least privilege becomes important.
Instead of:
1
Allow *
the application should receive only the permissions required for its job.
For example:
1
2
3
4
5
backend
|
+--> s3:GetObject
|
+--> specific bucket
rather than:
1
2
3
backend
|
+--> AdministratorAccess

6. The Trust Policy Is Just as Important

One mistake I found particularly interesting is focusing only on the permissions policy.
An IAM role also has a trust policy.
For EKS Pod Identity, AWS documents a trust relationship using the:
1
pods.eks.amazonaws.com
service principal, with actions including:
1
2
sts:AssumeRole
sts:TagSession
([AWS Documentation][2])
A simplified example is:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
This leads to an important distinction:
1
2
3
4
5
6
7
IAM Permissions Policy
|
+--> What can the role DO?

IAM Trust Policy
|
+--> Who can ASSUME the role?
Both matter.

7. Creating the Pod Identity Association

AWS provides an EKS API/CLI operation to create the association.
For example:
1
2
3
4
5
aws eks create-pod-identity-association \
--cluster-name my-cluster \
--role-arn arn:aws:iam::123456789012:role/backend-s3-role \
--namespace default \
--service-account backend-sa
The association connects:
1
2
3
4
5
6
7
EKS Cluster
|
+--> Namespace: default
|
+--> ServiceAccount: backend-sa
|
+--> IAM Role
AWS notes an important difference from IRSA: with EKS Pod Identity, you don't add an IAM role annotation to the ServiceAccount. The association is configured through EKS. ([AWS Documentation][3])

8. Don't Forget the Pod Identity Agent

Another common troubleshooting point is the EKS Pod Identity Agent.
The agent is required for EKS Pod Identity and runs as a DaemonSet on eligible nodes. ([AWS Documentation][1])
So the complete picture becomes:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
EKS Cluster
|
+-------------+-------------+
| |
v v
ServiceAccount Pod Identity Agent
| |
+-------------+-------------+
|
v
IAM Role
|
v
AWS APIs
If the identity association exists but the required agent is not available, the expected credential flow can fail.

9. My Troubleshooting Workflow

When I see:
1
AccessDenied
I would avoid immediately changing IAM policies.
Instead, I would walk through the chain systematically.

Step 1 — Is the Pod running?

1
kubectl get pods
If the Pod isn't healthy, fix the Kubernetes problem first.

Step 2 — Which ServiceAccount is the Pod using?

1
2
kubectl get pod <pod-name> \
-o jsonpath='{.spec.serviceAccountName}'
This is a very useful check.
You might discover that the application is actually using:
1
default
instead of:
1
backend-sa

Step 3 — Does the Pod Identity association exist?

1
2
aws eks list-pod-identity-associations \
--cluster-name my-cluster
AWS specifically recommends checking the Pod Identity association when troubleshooting workloads that cannot make AWS API calls. ([AWS Documentation][4])

Step 4 — Does the IAM role have the correct trust policy?

Check the role:
1
2
aws iam get-role \
--role-name backend-s3-role
Look for the expected Pod Identity trust relationship.

Step 5 — Does the IAM policy allow the requested action?

For example, if the application is attempting:
1
s3:GetObject
but the policy only allows:
1
s3:ListBucket
the application can still receive:
1
AccessDenied
The action, resource, and conditions all matter.

Step 6 — Check CloudTrail

If the IAM configuration looks correct, CloudTrail can help identify denied AWS API calls and the principal involved.
This turns:
1
"It doesn't work."
into something much more concrete:
1
2
3
4
5
Which AWS API?
Which principal?
Which action?
Which resource?
Why was it denied?

10. The Most Useful Mental Model

This is the model I now use when debugging EKS-to-AWS access:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Is the Pod healthy?
|
v
Which ServiceAccount?
|
v
Is Pod Identity configured?
|
v
Is the agent available?
|
v
Which IAM role?
|
v
Can the role be assumed?
|
v
Does the policy allow it?
|
v
Does the resource allow it?
|
v
Check CloudTrail
Instead of treating AccessDenied as one problem, break it into layers.

11. Pod Identity vs. IRSA

EKS also supports IAM Roles for Service Accounts (IRSA).
Both approaches can provide fine-grained AWS permissions to Kubernetes workloads. ([AWS Documentation][5])
The conceptual difference is:
1
2
3
4
5
6
7
8
9
IRSA

Pod
|
+--> ServiceAccount
|
+--> OIDC
|
+--> IAM Role
while Pod Identity uses:
1
2
3
4
5
6
7
8
9
EKS Pod Identity

Pod
|
+--> ServiceAccount
|
+--> Pod Identity Association
|
+--> IAM Role
One notable difference is that EKS Pod Identity does not require configuring an OIDC identity provider for each cluster, and the association is managed through EKS rather than through a ServiceAccount annotation. ([AWS Documentation][1])
This doesn't mean every existing EKS environment should automatically migrate from IRSA.
The right choice depends on the architecture, existing configuration, operational requirements, and organizational standards.

12. One More Important Distinction: Human Access vs. Pod Access

There is another identity problem in EKS that is easy to mix up with Pod Identity.
Suppose I want to run:
1
kubectl get pods
That is different from an application inside a Pod running:
1
aws s3 ls
The first concerns access to the Kubernetes API.
The second concerns access from the workload to AWS APIs.
Conceptually:
1
2
3
4
5
6
7
8
9
10
Human / CI system
|
v
AWS IAM
|
v
EKS API
|
v
Kubernetes resources
versus:
1
2
3
4
5
6
7
8
9
10
11
12
13
Pod
|
v
ServiceAccount
|
v
EKS Pod Identity
|
v
IAM Role
|
v
AWS Service
EKS access entries are one mechanism AWS provides for granting IAM principals access to the Kubernetes API. ([AWS Documentation][6])
Keeping these two paths separate prevents a lot of confusion.

13. What This Taught Me

The biggest lesson wasn't a particular CLI command.
It was learning to stop thinking of:
1
AccessDenied
as a single problem.
In an EKS environment, I now ask:
1
2
3
4
5
6
7
8
9
10
11
12
13
Who is making the request?

Which identity is being used?

Which IAM role is involved?

Can that role be assumed?

What permissions does it have?

What resource is being accessed?

Is another policy or condition blocking the request?
That turns troubleshooting from trial-and-error into a repeatable process.

Final Takeaway

A running Pod doesn't automatically mean that the application inside the Pod is authorized to access AWS resources.
The complete chain matters:
1
2
3
4
5
6
7
8
9
10
11
Pod
↓
ServiceAccount
↓
EKS Pod Identity
↓
IAM Role
↓
IAM Policy
↓
AWS Service
Once this chain becomes familiar, EKS IAM problems become much easier to investigate.
And perhaps the most useful debugging question is simply:
"Which identity is actually making this AWS API call?"
That question often gets you much closer to the real problem than changing IAM policies at random.

What I’d love to discuss

For people running workloads on EKS:
What has caused the most confusing AccessDenied issue in your experience?
Was it:
  • the wrong ServiceAccount?
  • an IAM trust policy?
  • missing permissions?
  • a Pod Identity association?
  • IRSA/OIDC configuration?
  • or something completely different?
I'd be interested to hear the failure mode that took you the longest to diagnose.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article