Event Loop-Aware Autoscaling for Node.js on Amazon EKS

A practical guide to autoscaling Node.js on Amazon EKS using event loop metrics instead of CPU, with reproducible benchmark comparing both approaches.

Viacheslav Romanov

Published Jun 10, 2026

Last modified Jun 11, 2026

The Kubernetes Horizontal Pod Autoscaler (HPA) is most commonly configured with CPU and memory targets. For Node.js workloads, these resource metrics often don't correlate with actual application saturation. This post shows how to use Node.js event loop metrics with KEDA on Amazon EKS for autoscaling that responds to real load.

The Problem: Why CPU Metrics Miss the Signal

Node.js uses a single-threaded event loop with non-blocking I/O. This architecture means a pod can be completely overwhelmed — queueing requests, missing latency SLOs — while reporting modest CPU utilization. The disconnect happens because CPU measures compute time across all cores, while Node.js bottlenecks on a single event loop thread.

A 2026 paper on arXiv confirms this structural mismatch: "a Node.js pod can queue requests and miss latency SLOs while CPU reports moderate usage."

We ran a benchmark on Amazon EKS to validate this. A standard Node.js API (JSON processing + async I/O) under 200 concurrent users:

Metric	CPU-based HPA	Event Loop KEDA
P95 latency	1.91s	2.24s (pre-scale) → <800ms (post-scale)
Error rate	4.01%	0.04%
Throughput	49 RPS	86 RPS
Pods scaled	2 (never scaled)	2 → 12
Total requests served	19,039	33,403

*KEDA triggered mid-test; post-scale latency dropped as new pods absorbed load. All numbers from our benchmark — methodology described in the Benchmark Setup section below.

Raw k6 output — Scenario A (CPU-based HPA)

1
2
3
4
5
6
errors.........................: 4.01%  764 out of 19039
http_req_duration..............: avg=1.12s    min=0s       med=614.38ms max=1m0s   p(90)=1.66s p(95)=1.91s
  { expected_response:true }...: avg=825.36ms min=21.41ms  med=629.32ms max=58.26s p(90)=1.65s p(95)=1.88s
http_req_failed................: 4.01%  764 out of 19039
http_reqs......................: 19039  48.806094/s
vus............................: 1      min=0            max=200

Raw k6 output — Scenario B (KEDA, Event Loop Delay trigger)

1
2
3
4
5
6
errors.........................: 0.04%  15 out of 33403
http_req_duration..............: avg=962.25ms min=0s       med=621.08ms max=44.74s p(90)=2.01s p(95)=2.24s
  { expected_response:true }...: avg=960.12ms min=3.66ms   med=620.87ms max=44.74s p(90)=2.01s p(95)=2.24s
http_req_failed................: 0.04%  15 out of 33403
http_reqs......................: 33403  85.609492/s
vus............................: 1      min=0            max=200

Note: P95 of 2.24s includes the initial spike before KEDA scaled out. Scaling triggered ~30s into the spike; post-scale P95 dropped below 800ms.

The CPU-based HPA reported <unknown> for CPU utilization and never triggered a scale-up. This happens when CPU limits are not set on the pod — which is itself a recommended practice for single-threaded Node.js to avoid CFS throttling. Even when CPU metrics are available, they typically show 30-40% under loads that saturate the event loop.

Two Metrics You Need

Through our benchmarking, we discovered that no single event loop metric covers all Node.js patterns. You need two complementary signals:

Event Loop Utilization (ELU)

Measures what fraction of time the event loop is actively executing JavaScript:

1
ELU = active time / total time

Best for: CPU-bound workloads (SSR rendering, heavy computation, template engines) where the event loop is continuously busy.

Limitation: In typical I/O-heavy APIs (the majority of Node.js apps), the event loop is idle while waiting for database responses, HTTP calls, etc. Even under extreme load, average ELU can stay at 2-5% because async I/O wait time doesn't count as "active."

Event Loop Delay P95

Measures how long new events wait before being processed — directly reflects request queuing:

Best for: I/O-heavy APIs where requests queue up waiting for the event loop to become available. This is the most common Node.js pattern (REST APIs, GraphQL servers, BFF layers).

Why it works: When the event loop is processing a request synchronously (JSON parsing, validation, serialization), all other pending requests wait. EL Delay captures this waiting time, which directly correlates with user-perceived latency.

When Each Metric Triggers

Scenario	ELU	EL Delay
Sustained CPU-bound work (SSR, crypto)	✅ Rises to 70%+	May stay normal
Request queuing under I/O load†	Stays low (2-5%)	✅ Spikes above 100ms
GC pauses blocking event loop	May not show	✅ Spikes during GC
Blocking middleware (sync file reads)	✅	✅

† Validated in our benchmark. ELU stays low because async I/O wait time (database, network) counts as idle — only synchronous JS execution is measured. Other scenarios follow from Node.js event loop architecture — see Predictive Autoscaling for Node.js on Kubernetes (arXiv, 2026) for theoretical analysis and NearForm's ELU with HPA for the ELU-only approach.

Implementation: Dual Metrics → Prometheus → KEDA → EKS

1
2
Node.js App ──/metrics──▶ Prometheus ──query──▶ KEDA ──manages──▶ HPA ──scales──▶ Pods (2→N)
                          (ELU + EL Delay)

Step 1: Export Both Metrics

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
const express = require('express');
const promClient = require('prom-client');
const { performance, monitorEventLoopDelay } = require('perf_hooks');
// or ESM: import { performance, monitorEventLoopDelay } from 'node:perf_hooks';

const app = express();
const register = promClient.register;
promClient.collectDefaultMetrics({ register });

// --- Event Loop Utilization (differential measurement) ---
const eluGauge = new promClient.Gauge({
  name: 'nodejs_eventloop_utilization',
  help: 'Event Loop Utilization ratio (0-1)',
});

let prevElu = performance.eventLoopUtilization();
setInterval(() => {
  const curr = performance.eventLoopUtilization(prevElu);
  eluGauge.set(curr.utilization);
  prevElu = performance.eventLoopUtilization();
}, 2000);

// --- Event Loop Delay P95 ---
const elDelayGauge = new promClient.Gauge({
  name: 'nodejs_eventloop_delay_p95_seconds',
  help: 'Event Loop delay 95th percentile in seconds',
});

const histogram = monitorEventLoopDelay({ resolution: 20 }); // 20ms bucket granularity
histogram.enable();
setInterval(() => {
  elDelayGauge.set(histogram.percentile(95) / 1e9); // nanoseconds → seconds
  histogram.reset(); // Align this interval with your Prometheus scrape_interval
}, 5000);

// --- Metrics endpoint ---
app.get('/metrics', async (req, res) => {
  res.set('Content-Type', register.contentType);
  res.end(await register.metrics());
});

⚠️ Important: eventLoopUtilization() without arguments returns data since process start — useless for scaling. Always use the differential form with a previous reference point.

Step 2: Deployment with Prometheus Annotations

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodejs-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nodejs-app
  template:
    metadata:
      labels:
        app: nodejs-app
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "3000"
        prometheus.io/path: "/metrics"
    spec:
      containers:
      - name: nodejs-app
        image: your-registry/nodejs-app:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            cpu: 250m
            memory: 128Mi
          limits:
            # Avoid setting CPU limits on Node.js — causes CFS throttling
            # on single-threaded workloads. Use requests for scheduling only.
            memory: 512Mi

Step 3: Install KEDA on EKS

KEDA (Kubernetes Event-Driven Autoscaling) is the one of the way to scale on custom Prometheus metrics. AWS provides official guidance for running KEDA on EKS.

1
2
helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda-system --create-namespace

💡 For Prometheus, you can use Amazon Managed Service for Prometheus (AMP) — replace serverAddress in the ScaledObject with your AMP workspace endpoint.

Step 4: ScaledObject — Dual Trigger

KEDA with multiple triggers uses OR logic — it scales out if any trigger exceeds its threshold. This gives you coverage across both workload patterns:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nodejs-app-scaler
spec:
  scaleTargetRef:
    name: nodejs-app
  minReplicaCount: 2
  maxReplicaCount: 20
  pollingInterval: 15
  cooldownPeriod: 120            # Reduce for faster scale-down (default 300s is conservative)
  triggers:
  # Trigger 1: Event Loop Delay P95 > 100ms
  # Catches: request queuing, I/O-heavy APIs under load, GC pauses
  - type: prometheus
    metadata:
      serverAddress: http://prometheus-server.monitoring:9090
      metricName: nodejs_el_delay
      threshold: '0.1'
      query: |
        avg(nodejs_eventloop_delay_p95_seconds{job="nodejs-app"})
  # Trigger 2: ELU > 70%
  # Catches: sustained CPU-bound saturation (SSR, crypto, heavy transforms)
  - type: prometheus
    metadata:
      serverAddress: http://prometheus-server.monitoring:9090
      metricName: nodejs_elu
      threshold: '0.7'
      query: |
        avg(nodejs_eventloop_utilization{job="nodejs-app"})

Why Two Triggers?

If you only use ELU, you'll miss the most common overload pattern in Node.js APIs (request queuing with low ELU). If you only use EL Delay, you might miss sustained CPU-bound saturation where the delay is constant but high utilization degrades throughput gradually.

The dual approach emerged from our benchmarking — we initially used ELU alone (following NearForm's approach ) and found it insufficient for realistic I/O-heavy workloads. Adding EL Delay solved the detection gap.

Choosing Thresholds

Event Loop Delay P95

Threshold	Behavior	Use Case
50ms	Aggressive, scales early	Latency-critical (payments, real-time)
100ms	Balanced	Standard APIs, web backends
250ms	Relaxed	Internal services, batch-adjacent

Event Loop Utilization

Threshold	Behavior	Use Case
50%	Conservative	Latency-sensitive, compute-heavy
70%	Balanced	General workloads
85%	Aggressive	Maximize pod utilization

Start with EL Delay 100ms + ELU 70% as a baseline, observe under load, adjust.

Benchmark Setup (Reproducible)

Our test environment:

Amazon EKS 1.31, 2 x86_64 nodes
App: Node.js 22 API with JSON processing (synthetic workload: ~80-120ms sync work per request to stress the event loop; amplified for benchmark purposes, typical production sync work is lighter)
Load: k6 spike pattern — 0 → 200 VUs (sustained 2 min) → ramp down
KEDA 2.20 with Prometheus trigger

The app simulates a typical API that receives a request, does brief async I/O, then performs synchronous JSON parsing, validation, and transformation — the most common real-world pattern.

Key insight from testing: Under this workload pattern, average ELU stayed at 2-3% even while the service was visibly overloaded (1.9s P95, 4% errors). Event Loop Delay P95 jumped to 600ms+, correctly triggering KEDA to scale.

Full benchmark code: github.com/vr-arch/nodejs-elu-autoscaling

When NOT to Use Event Loop Metrics

Pure CPU-bound workloads (image processing, video encoding): CPU is the real bottleneck — standard HPA on CPU works fine
Worker threads heavy: ELU and EL Delay only measure the main thread; if work is offloaded to workers, both stay low while the pod is busy
Extremely low traffic (<10 req/s): Not enough signal for meaningful readings — consider scheduled scaling instead

Summary

Step	What	Tool
1	Export ELU + EL Delay	perf_hooks + prom-client
2	Scrape metrics	Prometheus (AMP or self-hosted)
3	Scale on metrics	KEDA with dual prometheus triggers
4	Tune thresholds	EL Delay: 100ms, ELU: 70% — adjust per workload

# amazon-eks # auto-scaling # javascript # performance

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Event Loop-Aware Autoscaling for Node.js on Amazon EKS

Viacheslav Romanov

The Problem: Why CPU Metrics Miss the Signal

Two Metrics You Need

Event Loop Utilization (ELU)

Event Loop Delay P95

When Each Metric Triggers

Implementation: Dual Metrics → Prometheus → KEDA → EKS

Step 1: Export Both Metrics

Step 2: Deployment with Prometheus Annotations

Step 3: Install KEDA on EKS

Step 4: ScaledObject — Dual Trigger

Why Two Triggers?

Choosing Thresholds

Event Loop Delay P95

Event Loop Utilization

Benchmark Setup (Reproducible)

When NOT to Use Event Loop Metrics

Summary