AWS Builder Center
Kubernetes Auto-Scaling in Action - HPA on Amazon EKS

Kubernetes Auto-Scaling in Action - HPA on Amazon EKS

One of the most powerful features of Kubernetes is its ability to automatically scale your application in response to real load. Instead of over-provisioning servers and paying for idle capacity, Kubernetes watches your metrics and adds or removes pods as demand changes. In this demo, I set up a Horizontal Pod Autoscaler (HPA) on an Nginx deployment running on Amazon EKS - then stressed the pods to watch them scale up live, and removed the load to watch them scale back down.

Head of DevOps

What is HPA?

HPA automatically adjusts the number of pod replicas in a Deployment based on observed metrics — typically CPU utilization. You define a target (e.g., keep average CPU below 30%) and HPA continuously adjusts replica count to maintain that target.
Low CPU → HPA scales DOWN → fewer pods (cost saving)
High CPU → HPA scales UP → more pods (performance)
HPA does not change the size of individual pods — it changes how many pods are running. That is the difference between Horizontal Scaling (more pods) and Vertical Scaling (bigger pods).



Architecture

HTTP Traffic
↓
AWS Load Balancer
↓
Nginx Pods (2 → 4 → 6 → 8 based on CPU)
↑
HPA watches CPU metrics via Metrics Server
↑
Metrics Server collects from Kubelet on each Node

Prerequisites

Metrics Server must be installed and running — HPA reads CPU metrics from it. This was installed in our cluster via the add-ons script in Demo 1:
kubectl get pods -n kube-system | grep metrics-server
# metrics-server-xxx 1/1 Running ✅

# Verify metrics are flowing
kubectl top nodes
kubectl top pods -n aiops-demo
Without Metrics Server, HPA will show  <unknown>  for targets and will never scale.



Step 1 — CPU Resource Requests (Required for HPA)

HPA calculates utilization as a percentage of the pod’s CPU request. Without a CPU request defined, HPA cannot compute a meaningful percentage and will not function.
The Nginx deployment was updated to include explicit resource requests and limits:
resources:
requests:
cpu: "100m" # HPA uses this as the baseline
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
 100m  means 100 millicores — one tenth of a CPU core. This is the minimum CPU the scheduler guarantees for the pod.



Step 2 — Create the HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: nginx-hpa
namespace: aiops-demo
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: nginx-deployment
minReplicas: 2
maxReplicas: 8
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 10 # scale out when avg CPU > 10%
The threshold was set to 10% (lowered from the initial 30%) to make scaling easier to trigger in a demo environment — Nginx is highly efficient and does not consume much CPU even under significant HTTP load.
kubectl apply -f nginx-hpa.yaml

kubectl get hpa -n aiops-demo
# NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS
# nginx-hpa Deployment/nginx-dep 3%/10% 2 8 2

Step 3 — Triggering Scale-Up with CPU Stress

HTTP load testing alone (using Apache Bench or hey) was not enough to push Nginx CPU above the threshold — Nginx handles static requests with very little CPU. The effective approach was to inject CPU stress directly into the running pods:
# Get pod names
POD1=$(kubectl get pod -n aiops-demo -l app=nginx \
-o jsonpath='{.items[0].metadata.name}')
POD2=$(kubectl get pod -n aiops-demo -l app=nginx \
-o jsonpath='{.items[1].metadata.name}')

# Install and run stress-ng inside each pod
kubectl exec $POD1 -n aiops-demo -- \
sh -c "apk add -q stress-ng && stress-ng --cpu 2 --timeout 120s &"

kubectl exec $POD2 -n aiops-demo -- \
sh -c "apk add -q stress-ng && stress-ng --cpu 2 --timeout 120s &"

Step 4 — Watching HPA Scale Up

With two terminals open — one watching HPA, one watching pods:
# Terminal 1
watch -n 2 kubectl get hpa -n aiops-demo

# Terminal 2
watch -n 2 kubectl get pods -n aiops-demo
HPA progression after CPU stress was applied:
TIME TARGETS REPLICAS EVENT
T+0:00 3%/10% 2 baseline
T+0:30 22%/10% 2 CPU rising
T+1:00 45%/10% 4 ← scale up triggered
T+1:30 60%/10% 6 ← scale up again
T+2:00 35%/10% 8 ← max replicas reached
The new pods came up in Running state within 15–20 seconds — Nginx is a lightweight image with fast startup time.



Step 5 — Scale Down After Load Removed

Once the stress timeout elapsed (120 seconds), CPU dropped back below the threshold. Kubernetes applies a cooldown period of 5 minutes before scaling down — this prevents rapid flapping when load fluctuates briefly.
TIME TARGETS REPLICAS EVENT
T+2:00 35%/10% 8 stress ending
T+3:00 8%/10% 8 cooldown active
T+5:00 2%/10% 6 ← scale down starts
T+7:00 1%/10% 4 ← continuing scale down
T+9:00 1%/10% 2 ← back to minimum ✅

HPA Scaling Formula

HPA uses this formula to calculate desired replicas:
desiredReplicas = ceil(currentReplicas × (currentCPU / targetCPU))

Example:
currentReplicas = 2
currentCPU = 45%
targetCPU = 10%

desiredReplicas = ceil(2 × (45/10)) = ceil(9) = 8
This is why HPA jumped directly toward max replicas when CPU was high — the math said 8 pods were needed.



Key Lessons

HTTP load alone may not trigger HPA for Nginx. Nginx is extremely efficient at serving static content. Even 100 concurrent connections with 100,000 requests barely moved the CPU needle. For realistic HPA testing, use CPU stress directly inside the pods or deploy a CPU-intensive application.
CPU requests are mandatory for HPA. Without  resources.requests.cpu  defined in the pod spec, HPA cannot calculate utilization percentages and will show  <unknown>  for targets. Always define CPU requests on any deployment that will use HPA.
Scale-down has a built-in delay. Kubernetes waits approximately 5 minutes before scaling down after load drops. This is intentional — it prevents instability when traffic fluctuates. Do not mistake this delay for HPA not working.
minReplicas protects against zero-pod scenarios. Even if CPU drops to 0%, HPA will never go below  minReplicas . Setting this to 2 ensures at least basic redundancy at all times.
Set maxReplicas based on node capacity. With one  t3.medium  node in this demo, 8 pods was feasible. In production, HPA should be paired with Cluster Autoscaler so that node capacity grows alongside pod count.
HPA + Cluster Autoscaler Together
HPA: scales pods within existing nodes
Cluster Autoscaler: adds nodes when pods cannot be scheduled

Together:
Traffic spike
↓
HPA: add more pods
↓
Nodes full → pods Pending
↓
Cluster Autoscaler: add new node
↓
Pending pods scheduled on new node ✅
This combination is what makes Kubernetes truly elastic — both layers scale automatically in response to demand.



Full Source Code

All YAML manifests used in this demo are available on GitHub:
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article