Kubernetes From Scratch, Part 4: Autoscaling and Health Probes

Part 3 of this series ended with me moving toward two things: autoscaling based on load instead of a fixed replica count, and health probes so the cluster actually knows when a Pod is ready to serve traffic. This entry covers both because it turns out they're more connected than I expected.

Up to this point, my Deployments specified a fixed number of replicas I'd pick 3, or 22 like I did on FoodRush, and that was that. But a fixed number is either wasteful at low traffic or insufficient at peak traffic. The obvious next step was Horizontal Pod Autoscaling (HPA) scaling replica count automatically based on actual load.

An HPA resource watches a metric most commonly CPU utilization against a target you set, like "keep average CPU at 50%." If Pods are running hotter than that, it scales replicas up; if usage drops, it scales back down, within a min/max range you define.

The part that wasn't obvious to me at first: HPA needs the Metrics Server running in the cluster to actually have CPU/memory data to act on. Without it, an HPA resource just sits there unable to make decisions similar to how an Ingress resource does nothing without a controller, a pattern I kept running into throughout this series.

Here's the connection I didn't initially see: scaling up means new Pods are starting constantly, and a brand-new Pod isn't instantly ready to serve traffic it might still be loading config, connecting to a database, or warming up. If the Service sends it traffic too early, requests fail.

Readiness probes solve this they tell Kubernetes "don't send this Pod traffic until it passes this check." Until a readiness probe succeeds, the Pod is excluded from the Service's load-balancing pool entirely.

Liveness probes solve a different problem they tell Kubernetes "this Pod is still running, but is it actually healthy, or silently stuck?" If a liveness probe keeps failing, Kubernetes kills and restarts the Pod, rather than leaving a zombie process accepting connections it can never properly answer.

My first attempt used the same endpoint and same timing for both probes. This caused a problem: my app had a slow startup (loading some initial data), so the liveness probe kept failing during startup and Kubernetes kept restarting the Pod in a loop it never got the chance to finish starting. The fix was adding an initialDelaySeconds buffer to the liveness probe, and realizing readiness and liveness don't have to check the same thing at the same cadence readiness can be stricter and more frequent, liveness more lenient and infrequent.

With HPA scaling Pods up and down constantly, and readiness/liveness probes correctly gating traffic and catching stuck Pods, the Deployment becomes genuinely self-managing scaling to load, routing around not-yet-ready Pods, and recovering from Pods that silently hang. That's a meaningfully different system than the fixed-replica, no-probes setup I started this series with.

I'm planning to bring this series into EKS on AWS next, since everything so far has been on a local cluster. I also want to connect the DevSecOps side I've built in my other projects scanning manifests, enforcing resource limits directly into this Kubernetes workflow.

If you're still running fixed replica counts with no probes: this is the next real upgrade. It's less about new commands and more about the cluster starting to make decisions on its own.