The half of the invoice nobody opens
Karpenter v1.13.0 shipped on June 10, and my feed has been wall-to-wall EKS cost advice ever since. Consolidate your nodes. Move stateless workloads to Spot. Switch to Graviton. Right-size everything. All of it is correct, and all of it is about compute.
Compute is the right place to start. It's usually the biggest line on the bill and the one you control most directly. But it's one half of the invoice. The other half is data transfer, and it survives every cost review because it never arrives as one big number. It arrives as a dozen small rows nobody scrolls down to read.
This is the order I actually work a noisy EKS bill, and the commands I use at each step.
What each byte costs
Three rates matter on a cluster in us-east-1 :
Line item | Rate | Where it comes from |
|---|
Cross-AZ transfer | $0.01/GB each direction | pods, services, or storage talking across Availability Zones |
NAT Gateway processing | $0.045/GB | any outbound byte routed through a NAT, on top of egress |
Internet egress | ~$0.09/GB (first 10 TB) | traffic leaving AWS to the internet |
The trap is that they compound. A pod that pulls an image from the internet through a NAT Gateway in a different AZ pays cross-AZ and NAT processing and egress on the same bytes.
Step 1: compute, because it is the biggest lever
Karpenter consolidation bin-packs pods onto fewer, cheaper nodes and lets idle capacity terminate. Spot covers anything that tolerates interruption. Graviton is mostly a node-pool change if your images already build for arm64 .
One honest note on the release that started the conversation: v1.13.0's changes are operational, not financial. Subnet-level IP tracking, configurable AMI and subnet refresh, custom IAM paths. The consolidation engine that makes Karpenter worth running has been there all along. Do all of this first, then look at the row underneath.
Step 2: measure the cross-AZ traffic before you argue about it
Cross-AZ transfer bills under a single usage type per region, <region>-DataTransfer-Regional-Bytes :
aws ce get-cost-and-usage \
--time-period Start=2026-06-01,End=2026-07-01 \
--granularity MONTHLY \
--metrics UnblendedCost UsageQuantity \
--filter '{"Dimensions":{"Key":"USAGE_TYPE","Values":["USE1-DataTransfer-Regional-Bytes"]}}' \
--group-by Type=DIMENSION,Key=LINKED_ACCOUNT
Regional transfer is metered on both ends, so one round trip between two AZs produces two line items, which is why the $0.01/GB rate is effectively $0.02/GB on every request and reply. Worth knowing before you read the answer: that number covers the whole account, not your cluster.
Getting from a bill line to a workload means asking the cluster where its endpoints actually live. EndpointSlices carry the zone per endpoint:
kubectl get endpointslices -l kubernetes.io/service-name=checkout \
-o jsonpath='{range .items[*].endpoints[*]}{.zone}{"\t"}{.targetRef.name}{"\n"}{end}'
Then look at where the callers sit, because a service spread across three zones only costs you when the traffic reaching it started somewhere else:
kubectl get nodes -L topology.kubernetes.io/zone
kubectl get pods -l app=web -o custom-columns=POD:.metadata.name,NODE:.spec.nodeName
"Isn't spreading across AZs the whole point?"
For availability, yes. I'm not arguing against multi-AZ. I'm arguing for knowing what it costs you, so it's a decision instead of an accident.
Every time a pod in us-east-1a calls a service whose endpoint happens to land in us-east-1b , that round trip costs $0.01/GB out and $0.01/GB back. No single call is expensive. For a chatty architecture (service mesh, sidecars, high-QPS internal APIs) the aggregate is a real line on the bill.
And a good share of the spread I find in the wild was never a decision. It's a Helm chart default and a constraint copied from a blog post:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: checkout
That block is load-bearing for a stateful service that must survive a zone failure. For an internal API behind three replicas of a caller sitting in one zone, it's a bill nobody traced back to a specific line of YAML.
Step 3: take the easy bytes off the NAT meter
NAT is usually the fastest money you'll find, because a large share of the traffic on the meter is talking to AWS itself and never needed to leave the network.
Gateway VPC endpoints for S3 and DynamoDB are free and take that traffic off the NAT Gateway entirely:
# S3 traffic through a Gateway VPC endpoint:
# no NAT processing charge, no cross-AZ charge.
resource "aws_vpc_endpoint" "s3" {
vpc_id = aws_vpc.main.id
service_name = "com.amazonaws.us-east-1.s3"
vpc_endpoint_type = "Gateway"
route_table_ids = aws_route_table.private[*].id
}
ECR needs interface endpoints, and that's the one that matters on a cluster scaling nodes up and down all day. Every new node pulls images, and by default every one of those pulls is metered at NAT rates:
locals {
interface_endpoints = [
"ecr.api",
"ecr.dkr",
"logs",
"sts",
]
}
resource "aws_vpc_endpoint" "interface" {
for_each = toset(local.interface_endpoints)
vpc_id = aws_vpc.main.id
service_name = "com.amazonaws.us-east-1.${each.value}"
vpc_endpoint_type = "Interface"
subnet_ids = aws_subnet.private[*].id
security_group_ids = [aws_security_group.endpoints.id]
private_dns_enabled = true
}
Warning: Interface endpoints are not free the way gateway endpoints are. You pay per AZ per hour plus a per-GB processing charge. On a busy cluster the ECR ones pay for themselves quickly. Provision a full set for a service that moves almost no traffic and you've made the bill worse.
Same query shape for the NAT side of the meter:
aws ce get-cost-and-usage \
--time-period Start=2026-06-01,End=2026-07-01 \
--granularity MONTHLY \
--metrics UnblendedCost UsageQuantity \
--filter '{"Dimensions":{"Key":"USAGE_TYPE","Values":["USE1-NatGateway-Bytes"]}}'
Note: Cost Explorer has no idea whether a gigabyte through the NAT was headed to S3, to ECR, or to an external API. It bills the same either way. VPC Flow Logs on the private subnets will tell you, once you have a day or two of data and query the top talkers.
Honest trade-offs
- Keeping traffic in-zone trades resilience for cost, and sometimes that's the wrong trade. For a stateful service where a zone failure means data loss, the transfer charge is the price of staying up. Pay it on purpose.
-
DataTransfer-Regional-Bytes is account-wide. It includes cross-AZ traffic from RDS, from ALBs, from anything else you run. Don't walk into a cost review calling that number the cluster's bill.
- Interface endpoints have their own hourly and per-GB cost. They're a NAT replacement for high-volume AWS-bound traffic, not a default to sprinkle on every VPC.
- What I got wrong for a long time: I treated data transfer as a rounding error, because no individual charge ever looked serious enough to chase. Whatever architecture came out of that, nobody ever argued about what it cost. That's the part I'd change.
Results
The order that has held up for me: compute first, because it's the biggest single lever and the most direct to pull. Cross-AZ second. It's the quiet one, and it needs measurement before it gets an opinion. NAT third, because gateway endpoints for S3 and DynamoDB are free and the ECR ones are close behind.
Run the two Cost Explorer queries above against one week of data. Look at how much of that data-transfer line is workloads talking to themselves across a zone boundary they never needed to cross.
Compute is the number you watch. Data transfer is the number that watches you.
Loading article