Before chasing high RPS: designing a Netty benchmark on Amazon EC2

My first Netty test on EC2 showed 1.06M req/s on the dashboard. Here is the reported setup, what I did not record, and how I would make the next run repeatable.

Saravana Priyan C

Published Oct 8, 2026

Last modified Oct 8, 2026

A fast server needs a credible test

A requests-per-second number is easy to quote and hard to interpret. Was the server doing useful work? Was the client fast enough? Did the test stop before the instance's network burst credits ran out?

This guide focuses on designing a credible high-throughput test for a plain-Java Netty HTTP server on Amazon EC2. The goal is to separate application limits from client and network limits, rather than treat a single RPS number as the whole result.

The question is simple: when throughput stops rising, how do we tell whether the limit is the Java application, the load generator, or the EC2 network?

The answer starts before the first request.

My first run: 1.06M req/s on the dashboard

I followed Visual Computer Science's video, "How to build a Java HTTP API that takes 1M requests per second," as a starting point for my Netty test on AWS. The video is inspiration, not evidence for my own results. [11]

My test used one 16-vCPU API server and two separate EC2 load-generator instances: three machines in total. Each generator ran wrk2 with 8 threads and an offered rate of 500,000 requests per second, giving a combined target of 1 million requests per second. I tested for seven minutes. The two generators were smaller than the server, but I did not record their exact sizes or the complete instance type names. I recorded the following dashboard values:

API request rate: 1.06 million requests per second.
JVM heap usage: approximately 100–180 MB.
GC activity: approximately 4 ms.
Memory allocation rate: approximately 1.25 GiB/s.
JVM thread activity: approximately 20 threads.

These are the dashboard values I observed, not a complete benchmark report. The request-rate value alone does not establish a seven-minute average, a maximum capacity, or a sustained rate with zero errors. The roughly 4 ms GC reading is not request latency; I did not record its exact metric definition. The heap range is not total machine or process memory. The combined offered rate is a configured target, not proof of the achieved successful rate. The difference between that target and the 1.06M dashboard reading needs to be reconciled using the dashboard metric definition and completed wrk2 output.

The run gives me a starting point, but I still need the setup and client-side output to explain the number. In particular, I cannot identify the limiting resource without server CPU, generator capacity, latency, errors, and network measurements. That is why the rest of this article focuses on making the test repeatable and the result interpretable.

What I did not record

Complete instance type names and generator sizes. My note "c6.4xlarge" is incomplete, so I cannot reliably identify the instance family or calculate the total fleet vCPUs.
Region, Availability Zones and whether traffic used private IPs; placement-group and ENA Express configuration.
Full wrk2 commands and completed output, including connections, exact -R arguments, successful completed rate, timeouts and errors. The 8 threads and 500,000 req/s target per generator are my reported settings.
The dashboard request-rate definition: whether 1.06M req/s was a point-in-time value, peak, interval average or sustained rate. I also did not record warm-up time.
Request latency percentiles, including p99, and server/generator CPU utilization.
ENA exceeded-counter deltas on all three machines.
Response body size, HTTP/TLS and keep-alive settings, and the source version tested.
JDK, Netty, kernel, ENA driver and wrk2 versions; the GC collector and exact meaning of the GC panel.

What I would do first

Check the applied vCPU quota in the intended Region and count the entire test fleet.
Use separate server and generator instances, private IPs, and one fixed HTTP workload.
Calculate the response-byte budget and query the exact instance size's baseline bandwidth.
Ramp traffic while collecting latency, errors, client/server CPU, and ENA counter deltas.
Repeat the sustained run before drawing conclusions about capacity.

1. Define what one request means

For a first baseline, use a deliberately small workload:

HTTP/1.1 over TCP, persistent connections, without request pipelining.
A fixed endpoint returning a known response with a correct Content-Length.
No database, external API, disk access, or per-request application logging.
Record request bytes, response headers and body size, connection count, and whether TLS is enabled.
Validate response status and content, not just the number of completed requests.

This baseline isolates HTTP parsing, Netty's pipeline, JVM execution, and networking. It does not represent a production API. Add TLS, serialization, and application work in separate test cases rather than hiding them inside one headline result.

Choose the latency and error limits before testing. Define success as the highest sustained rate that meets those limits, not the largest number the client briefly prints.

2. Check the Region's vCPU quota before choosing an instance

EC2 On-Demand quotas are regional and counted in vCPUs. C-family instances use the Running On-Demand Standard instance quota. AWS currently lists a default of 5 vCPUs for that category, but an account's applied quota may differ. Check Service Quotas in the intended Region rather than assuming the published default is your actual allowance. [1]

Count the whole test fleet. For example, a 32-vCPU server plus two 16-vCPU generators needs 64 vCPUs of available quota, before accounting for other running Standard instances. This is quota arithmetic, not a recommended starting size.

Request any needed increase before the test. A quota increase does not guarantee that the chosen instance type is available in an Availability Zone. On a small allowance, start with a smaller functional test; do not call it a capacity benchmark for a larger machine.

3. Compare exact instance sizes, not family headlines

C6i and C6in make a useful comparison because they offer compute-focused and network-focused options within a related Intel-based generation. They are comparison candidates, not a claim that these are the best or newest EC2 instances for every workload. [2]

At the 4xlarge size, both have 16 vCPUs and 32 GiB of memory. AWS lists network bandwidth of up to 12.5 Gbps for c6i.4xlarge and up to 50 Gbps for c6in.4xlarge. The words "up to" matter. The C6in family's 200 Gbps headline belongs to its largest sizes, not every C6in instance. [2]

AWS documents baseline bandwidth and network I/O credits for burst-capable instances. A brief run can benefit from credits and tell a different story from a sustained run. Query the exact size's baseline and record it with the result. [3]

This read-only AWS CLI query follows the fields in AWS's networking guide:

1
2
3
4
5
aws ec2 describe-instance-types \
  --region <your-region> \
  --instance-types c6i.4xlarge c6in.4xlarge \
  --query 'InstanceTypes[].[InstanceType,VCpuInfo.DefaultVCpus,NetworkInfo.NetworkPerformance,NetworkInfo.NetworkCards[0].BaselineBandwidthInGbps]' \
  --output table

Replace the Region placeholder first. Use a current AWS CLI if the baseline field is missing. Keep CPU, memory, AMI, JDK, and application settings as similar as possible across comparisons, while documenting any remaining differences. The advertised bandwidth alone does not predict RPS or packet-per-second capacity.

4. Do the network math before setting an RPS target

Suppose the target is 2 million requests per second and each response body is 1 KiB:

1
2
3
2,000,000 requests/s × 1,024 bytes × 8 bits/byte
= 16,384,000,000 bits/s
= 16.384 Gbps of response body alone

That is a calculated lower bound, not a measurement. HTTP headers, TCP/IP headers, Ethernet overhead, acknowledgments, and retransmissions add traffic. Request traffic must be budgeted separately in the inbound direction. AWS's instance bandwidth specification applies to inbound and outbound independently. [3]

A tiny response changes the likely bottleneck. Low payload bandwidth does not guarantee spare network capacity: EC2 also has packet-rate and connection-tracking allowances. Do not assume one request equals one packet or guess a size's PPS limit from its Gbps figure. [3][4]

Topology matters too. AWS documents a 5 Gbps single-flow limit between instances outside the same cluster placement group, with higher limits possible under specific configurations. An aggregate bandwidth headline is not a promise for one TCP connection. [3]

For the first test, put server and generators in the same Availability Zone, use private IPs, and restrict access to the test machines. This removes internet-path variability and avoids accidentally testing a public endpoint. Document placement groups or ENA Express if you use them; they change the experiment.

5. Prove that the load generator is not the ceiling

Run the generator on separate instances, not on the Netty server. Collect client-side CPU and network data with the same care as server-side data.

Increase the offered rate in steps. At every step, record the offered rate, achieved successful rate, errors, timeouts, latency distribution, and both sides' resource use. If the client cannot generate the requested load, label the point as generator-limited. A requested rate is not proof that the server received it.

At a suspected plateau, hold the workload and server configuration fixed and add generator capacity. If successful throughput rises, the earlier result was not evidence of the server's ceiling. If it stays flat, investigate server and network metrics before drawing a conclusion.

A constant-rate generator such as wrk2 is useful for HTTP tests because it addresses coordinated omission: a client that slows its request issuance during server stalls can understate the latency impact of those stalls. wrk2 has its own implementation and timing limits, so pin its version and record its configuration. It does not make generator capacity unlimited. [5]

6. Read ENA counters alongside RPS

On ENA-enabled Linux instances, inspect network statistics with ethtool. Identify the relevant interface first rather than assuming its name:

1
2
3
ip -br link
# Replace <interface> with the interface used for test traffic.
ethtool -S <interface> | grep -E 'allowance|pps'

The ENA metrics to watch include: [4]

bw_in_allowance_exceeded: packets queued or dropped after exceeding inbound bandwidth allowance.
bw_out_allowance_exceeded: packets queued or dropped after exceeding outbound bandwidth allowance.
pps_allowance_exceeded: packets queued or dropped after exceeding the bidirectional PPS allowance, with driver-specific fragmentation caveats documented by AWS.
conntrack_allowance_exceeded: packets dropped when the connection-tracking allowance prevents new connections.

Read the exceeded counters before and after each measured interval and calculate deltas. They are cumulative since the last driver reset. A nonzero lifetime value alone does not show that this particular run hit a limit. Collect them on generators as well as the server.

If a bandwidth counter rises at the throughput plateau, inspect the traffic budget. If the PPS counter rises while byte throughput looks modest, investigate packet rate. If connection tracking is exceeded, inspect connection churn and reuse.

A zero delta does not clear every possible network bottleneck. Correlate it with per-core CPU, retransmissions, errors, JVM allocation and GC activity, and client capacity. The CloudWatch agent can also publish ENA metrics for a timeline. [4]

7. Tune Netty with one hypothesis at a time

Keep the first configuration boring and reproducible. Pin the Netty and JDK versions, publish the pipeline setup, and separate correctness checks from performance runs.

Avoid blocking work on the event-loop threads. Track buffer ownership carefully: Netty uses reference-counted objects, and the component that consumes a reference-counted message generally has responsibility for releasing it. Netty's guide explains the ownership rules and exceptions. [6][7]

On Linux, Netty's native epoll transport is a reasonable later comparison against Java NIO. It requires the matching native dependency and supported platform; it is not a magic RPS switch. Record the transport rather than letting it become an invisible difference between runs. [8]

Change one factor at a time: worker count, connection count, response size, transport, or a JVM setting. Keep a baseline so a change can be reversed. If several settings change together, a higher number cannot tell us which change helped.

8. Make the run safe, repeatable, and affordable

Before generating traffic, read the current Amazon EC2 Testing Policy. It distinguishes legitimate network stress tests from traffic intended to overwhelm infrastructure, and says AWS-hosted endpoints should be in the local Region. Do not treat this article as blanket permission for flooding or DDoS simulation. [9]

Use only systems you own or have permission to test. Limit security-group access, keep a stop procedure ready, and ramp traffic gradually. Estimate instance-hours, storage, monitoring, and any transfer charges before launching the fleet. Billing alerts help track spending, but the test still needs an explicit duration and teardown plan. AWS's load-testing guidance calls out cost planning and service quotas. [10]

For each configuration:

Check correctness at low load.
Warm up the JVM and observe when behavior stabilizes.
Measure several steady intervals, with enough duration to distinguish network bursting from sustained behavior.
Repeat the run and report variability, not only the best sample.
Terminate test instances and remove temporary resources when done.

What a credible result should contain

Publish enough detail for another builder to understand the number:

Region, Availability Zone relationship, exact instance types and counts, network baseline, and topology.
AMI, kernel, ENA driver, JDK, Netty, generator versions, and source commit.
HTTP/TLS settings, response size, connection count, warm-up, duration, and offered load.
Successful throughput, errors, timeouts, latency percentiles, and run-to-run variation.
Client and server CPU/network data, GC observations, and ENA counter deltas.
The observed bottleneck, supporting evidence, and what remains uncertain.

The useful outcome is not "Netty is fast." It is "under these conditions, this was the sustained successful rate, these were the latency and error limits, and this evidence explains where capacity stopped increasing."

A benchmark that asks the right question is worth more than a large number we cannot explain. Start with a repeatable workload, measure both ends, and let the evidence tell you what to test next.