![[Experiment] Three Ways to Connect AWS Tokyo and Osaka over a Closed Network — How Much Does Speed Actually Differ?](https://prod-assets.cosmic.aws.dev/a/3GODUgu0WUkyvRwFOy61jwJ10Ho/Top-.webp)
[Experiment] Three Ways to Connect AWS Tokyo and Osaka over a Closed Network — How Much Does Speed Actually Differ?
I built VPC Peering, Transit Gateway, and PrivateLink between Tokyo and Osaka on real AWS infrastructure and measured latency and throughput with actual numbers. The fastest method for latency isn't the fastest for throughput — and getting a clean measurement turned out to be its own battle.
1. Introduction
Multi-region DR (Disaster Recovery) designs and workload latency requirements often call for connecting regions in the same partition — Tokyo (ap-northeast-1) and Osaka (ap-northeast-3), for example — over a closed network that never touches the public internet. AWS offers several ways to do this, each with its own characteristics, constraints, and cost structure.
I had already built multi-region setups with Transit Gateway before, but never actually compared how fast they are against the alternatives, so I decided to build all three and measure the real difference. This article covers the following three approaches, all built on my own personal AWS account. I measured latency and throughput within an AZ, between AZs in the same region, and across regions, and this post summarizes the differences, the likely causes, and how to choose between them.
- VPC Peering (inter-region)
- AWS PrivateLink (cross-region)
- AWS Transit Gateway (inter-region peering)
Before you read on, take a guess at which one turns out to be the fastest.
Some of you are probably already thinking "what about Site-to-Site VPN?" I left that (and Direct Connect) out of the actual measurements — I'll explain why shortly.
This post assumes a working understanding of AWS VPC networking basics (subnets, route tables, security groups), and is aimed at engineers who want to know the practical trade-offs of multi-region design.
2. Overview of the experiment
2.1 Paths under test
| Lane | Path | Connectivity |
|---|---|---|
| A | Tokyo-1a ↔ Tokyo-1a | Same AZ (baseline) |
| B | Tokyo-1a ↔ Tokyo-1c | Cross-AZ, same region (baseline) |
| C | Tokyo-1a ↔ Osaka-3a | VPC Peering |
| D | Tokyo-1a ↔ Osaka-3a | Transit Gateway inter-region peering |
| E | Tokyo-1a ↔ Osaka-3a | PrivateLink (cross-region) |
A and B are baselines so I can separate "the cost of crossing regions" from "the cost of crossing AZs," giving a physical baseline for how far apart Tokyo and Osaka really are before looking at the differences between C through E.
2.2 Why I didn't measure Site-to-Site VPN
A direct VPC-to-VPC Site-to-Site VPN requires a Virtual Private Gateway and Customer Gateway on both sides, with the Customer Gateway running software VPN (e.g. strongSwan) on an EC2 instance with an EIP. That's a bit off-topic for an article focused on VPC-to-VPC connectivity using nothing but managed AWS services, and the added complexity doesn't buy much new insight — the effect of IPsec overhead, for instance, is already well documented elsewhere — so I skipped it this time.
As for Direct Connect — you already know why. It's not a service you use to connect VPCs to each other, and it's definitely not something you set up just to run a personal benchmark.
3. Technical details of each approach
3.1 VPC Peering
- Connects two VPCs one-to-one. No transitive peering (if A is peered with B, and B with C, A still can't reach C). As VPCs grow you need a full mesh — N VPCs require N(N-1)/2 peering connections.
- Routing is static routes added to each VPC's route table. There's no automatic route propagation.
Inter-region VPC Peering has been available for the Tokyo region since 2018.
3.2 AWS PrivateLink (cross-region)
- PrivateLink exposes a service behind an NLB (Network Load Balancer) to other VPCs, one-way, through a VPC Endpoint Service.
- Until recently, an Interface Endpoint could only reach an Endpoint Service in the same region. Cross-region connectivity went GA in November 2024.
One thing I think is particularly important is Private DNS behavior. Same-region PrivateLink lets
PrivateDnsEnabled: true resolve the service by name via AWS's internal DNS alone, but cross-region endpoints don't support automatic private DNS resolution — you have to explicitly resolve the endpoint-specific DNS name (something like vpce-xxxx.vpce-svc-xxxx.ap-northeast-3.vpce.amazonaws.com).This was also my first time connecting two regions with PrivateLink, and I ran into three constraints while building it.
Constraint 1: An NLB can't have a UDP listener unless IPv6 source NAT is enabled
A
TCP_UDP listener on the NLB behind PrivateLink was rejected with Network Load Balancers used with AWS PrivateLink cannot have TCP_UDP listeners when 'EnablePrefixForIpv6SourceNat' is set to 'off'. A standalone UDP listener produced the same error — an NLB behind PrivateLink simply can't have a UDP listener unless IPv6 source NAT (EnablePrefixForIpv6SourceNat) is enabled, which requires IPv6 CIDRs and a dualstack NLB. That was out of scope here, so I measured PrivateLink (path E) over TCP only; paths A-D were measured over both TCP and UDP, so keep that asymmetry in mind in section 6.Constraint 2: An NLB doesn't forward ICMP
I originally planned to measure latency with ICMP
ping too, but a sanity check showed 100% packet loss. An NLB behind PrivateLink is an L4 pass-through that simply doesn't forward L3 protocols like ICMP at all. That's why I switched to TCP-connect-time-based latency measurement across all paths (section 5) — an annoying thing to trip over, but a good hands-on lesson in PrivateLink's one-way, protocol-limited design.Constraint 3: Cross-region support requires the provider-side NLB to span at least 2 AZs
Running
aws ec2 modify-vpc-endpoint-service-configuration --add-supported-regions failed with Services must support at least 2 availability zones for cross region. Enabling cross-region PrivateLink requires the provider-side NLB to span at least two AZs (I had initially placed it in just one). Adding a second, NLB-only subnet in another AZ fixed it.3.3 AWS Transit Gateway (inter-region peering)
- Hub-and-spoke connectivity that lets you manage many VPCs centrally.
- Cross-region connectivity uses a TGW peering attachment.
- Peering attachments don't support route propagation — unlike VPC attachments, you must manually add a static route to the peer's CIDR in the TGW route table.
- You can't add a route while the attachment is
pending-acceptance. The order is: create → accept → confirmavailable→ add route.
3.4 MTU differences
I assumed inter-region paths (VPC Peering and TGW) were capped at an MTU of 1500, but measuring with
ping -M do -s <size> (the Don't Fragment flag) gave a different result.| Path | MTU (measured) |
|---|---|
| Same region (intra-AZ / cross-AZ) | 9001 |
| Cross-region VPC Peering | 8500 |
| Cross-region TGW peering | 8500 |
| Cross-region PrivateLink | Couldn't measure (the NLB doesn't forward ICMP) |
I tried the same method on path E too, but it hit 100% packet loss even at a 1200-byte payload — consistent with constraint 2 above.
It turns out a March 2025 update added Jumbo Frame support (up to 8500 bytes) to inter-region VPC peering, so my 1500-byte assumption predated this update. The official AWS documentation now states clearly: "The MTU for inter-region VPC peering connections is 8500 bytes."
3.5 Encryption
For both VPC Peering and TGW peering, cross-region traffic is automatically encrypted with AES-256 at AWS's network infrastructure layer — no extra configuration required, and a quietly strong selling point.
| Method | Cross-region encryption |
|---|---|
| VPC Peering (inter-region) | AES-256 (automatic, network layer) |
| TGW peering (inter-region) | AES-256 (automatic, network layer) |
| PrivateLink (cross-region) | No explicit statement of the encryption method in the official docs (it traverses the AWS backbone, but I couldn't find wording as explicit as the "AES-256" statements for VPC Peering/TGW) |
3.6 Cost comparison
Cost structure varies quite a bit between the three methods. Figures below use actual unit prices for the Tokyo region (ap-northeast-1).
| Item | Unit price (Tokyo region) |
|---|---|
| VPC Peering connection itself | Free (no hourly charge) |
| Transit Gateway VPC attachment | $0.07/hour |
| Transit Gateway peering attachment | $0.07/hour |
| Transit Gateway data processing | $0.02/GB |
| Interface VPC Endpoint | $0.014/hour (per AZ) |
| VPC Endpoint data processing | $0.01/GB (up to 1PB/month) |
| Network Load Balancer | $0.0243/hour |
| NLB Capacity Unit (NLCU) | $0.006/hour (varies with usage) |
| Inter-region data transfer (Tokyo→Osaka, billed at the source) | $0.09/GB |
Here's the fixed monthly cost of keeping the connection up around the clock (the part independent of data volume, at 730 hours/month), excluding the pay-per-GB transfer charge:
| Method | Breakdown | Fixed monthly cost |
|---|---|---|
| VPC Peering | The connection itself is free | $0 |
| Transit Gateway | 4 attachments (VPC attachments on both sides, plus the peering attachment) × $0.07/hour × 730 hours | ~$204 |
| PrivateLink | 1 NLB × $0.0243/hour × 730 hours ≒ $17.7 + Interface Endpoint across 2 AZs × $0.014/hour × 730 hours ≒ $20.4 | ~$38 |
When data volume and connection count are low, VPC Peering is by far the cheapest. TGW's fixed cost buys the scale benefit of centrally managing many VPCs, and PrivateLink sits in between. The tipping point, in my view, is the moment a full mesh of Peering connections becomes unmanageable — that's when TGW's fixed cost starts to make sense.
4. Building the test environment
I built all of the infrastructure with CloudFormation, split across the following stacks: a Tokyo network stack (VPC, subnets/EC2 across 2 AZs, SSM interface endpoints, S3 results bucket), an Osaka network stack (same, in 1 AZ), a VPC Peering stack (the connection plus static routes in both directions), Transit Gateway stacks (both regions' TGWs, the inter-region peering, and the routes added after acceptance), and PrivateLink stacks (Osaka's internal NLB + VPC Endpoint Service, and Tokyo's Interface Endpoint).
4.1 CloudFormation's cross-region limitation
CloudFormation's
Export/Fn::ImportValue only works within a single region. So after deploying the Tokyo and Osaka network stacks, I pulled the Outputs with describe-stacks and passed them explicitly to the next stack via --parameter-overrides:1
2
3
4
5
6
7
8
OSAKA_VPC_ID=$(aws cloudformation describe-stacks \
--region ap-northeast-3 --stack-name osaka-network \
--query "Stacks[0].Outputs[?OutputKey=='VpcId'].OutputValue" --output text)
aws cloudformation deploy \
--region ap-northeast-1 --stack-name connectivity-peering-connection \
--template-file connectivity-peering-connection.yaml \
--parameter-overrides TokyoVpcId=$TOKYO_VPC_ID OsakaVpcId=$OSAKA_VPC_ID4.2 Gotcha: a route table can't hold two routes to the same destination
I only noticed this after deploying: adding both the VPC Peering route and the TGW route to the Tokyo route table at the same time failed with
The route identified by 10.1.0.0/16 already exists.Obvious in hindsight — a route table can only hold one route per destination CIDR, and both VPC Peering and TGW target the peer VPC's entire CIDR as the destination, so the two routes can't coexist in the same table.
I split the VPC-side static routes into two small, independent stacks — one for Peering, one for TGW — and deployed only one at a time depending on which path I was measuring. PrivateLink doesn't route to the VPC CIDR at all (it reaches the Interface Endpoint's ENI directly), so it's exempt from this constraint and stays active alongside either one at all times.
5. Measurement methodology
I evaluate results using p50/p95/p99 percentiles rather than a simple average, since an average is sensitive to outliers and momentary congestion and can mask the real difference between methods.
- Latency: I originally planned ICMP
ping, but since ICMP doesn't pass through the NLB on the PrivateLink path, I switched to TCP-connect-time-based measurement across all paths. For each path, I perform 300 TCP three-way handshakes against iperf3's listening port (5201), spaced 0.2 seconds apart, recording connection time as the RTT equivalent. Timeout is 0.4 seconds (well below Linux's ~1-second SYN retransmit, so a timeout is treated as a loss rather than a slow RTT). - Throughput:
iperf3for 30 seconds with-i 1(1-second interval reporting), across three patterns — single TCP (-P 1), 4 parallel TCP (-P 4), and UDP (-u, for jitter/loss) — three times each, pooling interval samples for p50/p95/p99 (roughly 270 samples per path). I originally planned 300 seconds × 3 runs, but scaled down after the network burst credit issue below, settling on 30 seconds.
5.1 Gotcha: network burst credit exhaustion
Measuring all paths at full volume (300 seconds × 3 runs), only path A — the first one I ran — showed the expected high throughput (4.96 Gbps single TCP, 12.4 Gbps with 4 parallel streams, 4.98 Gbps UDP). Every path after that, B through E, got stuck around 0.77 Gbps for both 4-parallel TCP and UDP.
The cause: network I/O burst credits on the
c6i.large instances I used. 45 minutes of continuous high-volume traffic on path A used up the credits, and subsequent paths ran with no time to recover — so what those paths measured wasn't the difference between connectivity methods, but the difference in timing of credit exhaustion.Shortening the run to 60 seconds plus a 5-10 minute cooldown fixed path A's own degradation, but the same degradation reappeared from path B onward. Network I/O burst credits apparently don't recover on a short timescale the way EC2 CPU credits do — they seem tracked as a cumulative budget since launch — so a cooldown of a few to ten minutes wasn't enough.
Shortening further to 30 seconds and re-running in reverse order (E → D → C → B → A), the first two (E, D) kept their high throughput, but the third (C) maintained high throughput for 16 seconds and then dropped to ~0.77 Gbps within a single second — catching the exact moment credits ran out. This confirmed the hypothesis: it's cumulative usage since launch draining the credit budget, not a particular method being slow, and the same behavior reproduced whether I measured forward or in reverse.
5.2 A note on this benchmark
Percentiles improve resilience to outliers, but this is a personal benchmark — each window is only a few minutes and doesn't capture variation by time of day or day of week. Section 6's throughput numbers eliminate the credit-exhaustion effect by measuring each path alone right after a restart, but they're still a single, short session. Treat these numbers as a general trend, not a guarantee.
5.3 Commands
Measurement is automated by pushing a script to EC2 via
aws ssm send-command:1
2
3
4
5
6
7
./scripts/run_measurement.sh <REGION> <CLIENT_INSTANCE_ID> <PATH_ID> <PEER_HOST> <RESULTS_BUCKET> [SKIP_UDP] [DURATION_SEC]
# Example: path A (intra-AZ, Tokyo)
./scripts/run_measurement.sh ap-northeast-1 $TOKYO_INSTANCE_A_ID A $TOKYO_INSTANCE_A2_IP $TOKYO_RESULTS_BUCKET 0 30
# Example: path E (PrivateLink, UDP unsupported so SKIP_UDP=1)
./scripts/run_measurement.sh ap-northeast-1 $TOKYO_INSTANCE_A_ID E $PL_ENDPOINT_DNS $TOKYO_RESULTS_BUCKET 1 30Internally: run
tcp_latency.py (TCP-connect-time probe, 300 iterations), then iperf3 (TCP -P 1 / -P 4 / UDP, each 30 seconds × 3 runs), then upload raw data to S3. To avoid credit exhaustion, my final process was to stop/start the EC2 instance right before each path and measure only that path.Since VPC Peering and TGW can't both route to the same CIDR (4.2), I redeploy just the corresponding VPC-side route stack before measuring paths C and D — for example, deleting the Peering route stacks and deploying
connectivity-tgw-vpcroute-tokyo/osaka.yaml to switch from Peering to TGW (and the reverse to switch back).The supplementary experiment in 6.2 (isolating the latency added by an NLB) takes advantage of the fact that the internal NLB on the PrivateLink provider side is also reachable directly at its private IP over VPC Peering or TGW — I simply pointed the target at the NLB's private IP instead of the instance. Finally, I sync the raw S3 data locally and compute p50/p95/p99 with a small aggregation script.
6. Results
6.1 Latency (TCP connect time, in ms)
| Path | p50 | p95 | p99 | max |
|---|---|---|---|---|
| A (same AZ) | 0.21 | 0.26 | 0.31 | 2.08 |
| B (cross-AZ) | 1.65 | 1.70 | 1.74 | 4.53 |
| C (VPC Peering) | 8.89 | 9.94 | 10.02 | 10.81 |
| D (TGW) | 11.01 | 12.33 | 12.77 | 17.95 |
| E (PrivateLink) | 13.32 | 14.58 | 16.00 | 19.65 |
Packet loss was 0% on every path (for E, I used the values from the 60-second protocol run, for reasons explained above). The values increase monotonically and cleanly from A through E — you can see the cost of crossing an AZ (A vs B), the cost of crossing a region (B vs C/D/E), and the additional cost specific to each connectivity method (C vs D vs E) stacking up.
6.2 Supplementary experiment: how many extra ms does going through an NLB add?
E came out slowest, but I wanted to isolate whether that's an inherent PrivateLink cost or simply the cost of an NLB. The internal NLB on the PrivateLink provider side is also reachable directly via VPC Peering or TGW, so I re-ran the TCP connect measurement against the NLB's private IP instead of the instance directly, keeping paths C and D otherwise unchanged (300 probes, a single run at a different time than 6.1, so the direct values below differ from 6.1 by roughly a hundred microseconds).
| Path | Target | p50 (ms) | Added by NLB |
|---|---|---|---|
| C | Instance directly | 8.73 | - |
| C+NLB | Via NLB | 9.94 | +1.21ms |
| D | Instance directly | 10.88 | - |
| D+NLB | Via NLB | 11.59 | +0.71ms |
| E | PrivateLink (cross-region) | 13.32 | - |
On both C and D, an NLB adds roughly +0.7 to +1.2ms. Comparing D+NLB (11.59ms) to E (13.32ms) leaves a gap of about +1.7ms. E's extra latency breaks down roughly like this:
- The cost of "crossing regions" itself (the 8-11ms seen in C and D)
- The cost of "going through an NLB" (roughly +0.7 to +1.2ms)
- A cost specific to PrivateLink's cross-region endpoint mechanism (roughly +1.7ms, likely ENI processing on the Interface Endpoint — including NAT — plus cross-region tunneling)
6.3 Throughput
As in 5.1, restarting the instance before each path and measuring it alone with a full credit balance gave clean numbers, free of the credit-exhaustion effect.
| Path | Single TCP (p50, Gbps) | 4-parallel TCP (p50, Gbps) | UDP (p50, Gbps) |
|---|---|---|---|
| A (same AZ) | 4.96 | 12.41 | 4.98 |
| B (cross-AZ) | 4.96 | 12.41 | 4.98 |
| C (VPC Peering) | 2.41 | 10.58 | 4.98 |
| D (TGW) | 2.50 | 9.67 | 4.98 |
| E (PrivateLink) | 2.36 | 9.99 | Not measured (see 3.2) |
Within the same region (A, B), both intra-AZ and cross-AZ show ~12.4 Gbps with 4 parallel TCP streams — essentially the instance type's rated capability. Across regions (C-E), 4-parallel TCP shows a real difference between methods: VPC Peering (10.58G) > PrivateLink (9.99G) > TGW (9.67G). Simply crossing a region costs almost 2Gbps of throughput versus staying in-region (the MTU difference — 9001 vs 8500 — is likely one contributing factor).
Single-stream TCP for C-E sits in a tight 2.36-2.50 Gbps range, with much less difference between methods than 4-parallel TCP. On a path with 8ms+ RTT, single-flow TCP throughput tends to be capped by the bandwidth-delay product, so with no parallelism these numbers are probably dominated more by RTT than by the connectivity method itself.
UDP sits around 4.98 Gbps for A-D with almost no difference by method. I sent
iperf3 -u -b 0 (unlimited), but the client's own send rate may have been the bottleneck, so this condition may not have been well suited to observing UDP bandwidth differences between methods.7. Discussion
7.1 Why latency lines up in this order
- A→B (~+1.4ms): the cost of crossing AZs within the same region — physically separate data centers, so this is reasonable.
- B→C (~+7.2ms): the cost of crossing regions itself, mainly physical distance and AWS backbone hop count.
- C→D (~+2.1ms): TGW's hub functionality adds extra hops (VPC attachment, peering attachment), so it's structurally natural for it to be slightly slower than VPC Peering's direct one-to-one connection.
- D→E (~+2.3ms): the 6.2 experiment shows +0.7 to +1.2ms of this is "going through an NLB," leaving ~+1.7ms as overhead specific to PrivateLink's cross-region endpoint mechanism.
Why does PrivateLink's own +1.7ms show up?
My own take: TGW and PrivateLink fundamentally do different kinds of work.
- TGW connects via L3 routing: TGW peering is an encrypted link between two TGWs, with packets forwarded straightforwardly hop by hop at L3. Neither VPC Peering nor TGW terminates the connection or repackages the packet along the way.
- PrivateLink connects as a proxy: an Interface Endpoint isn't just a path to the peer VPC — it includes bidirectional NAT so communication still works even with overlapping CIDR ranges.
I think this difference in what's actually happening is why PrivateLink adds that extra 1.7ms.
7.2 Relationship to MTU
The MTU for cross-region VPC Peering/TGW peering is 8500 versus 9001 in-region — only a ~500-byte gap. Yet the clean 6.3 measurements show a clear gap between same-region (A, B: ~12.4 Gbps) and cross-region (C-E: 9.7-10.6 Gbps) paths. A gap of nearly 2 Gbps feels like more than MTU alone can explain, so I suspect physical distance and hop-count overhead between regions is also at play.
7.3 Network burst credits are hard to work around
Throughput measurement was, by far, the hardest part of this exercise. Burstable instance types like
c6i.large manage network bandwidth internally via a credit system, so run duration, run order, and cooldowns can produce wildly different numbers on the exact same path. Shortening the run plus a cooldown wasn't enough to remove the first-run advantage; I only resolved it by restarting the instance before every single path and measuring alone. Worth keeping in mind for this kind of benchmark:- Use an instance type with guaranteed (non-burstable) bandwidth (I skipped this for cost reasons)
- Provision a fully independent instance per path, or restart immediately before each measurement
7.4 Why does throughput differ between methods too?
4-parallel TCP orders as VPC Peering (10.58G) > PrivateLink (9.99G) > TGW (9.67G) — the opposite of the latency order, where TGW beats PrivateLink.
My guess: TGW's extra hops (7.1) plus its role as shared hub infrastructure aggregating many VPCs/connections likely give it a lower throughput ceiling than a simple one-to-one connection. PrivateLink adds latency via NAT and the NLB, but the NLB itself is built for high throughput, so it doesn't end up as disadvantaged as TGW on the throughput ceiling (speculation on my part).
8. Choosing an approach
Picking VPC Peering just because it's fastest on latency would be premature — a real architecture choice weighs speed, structure (operational manageability, future extensibility), and cost together.
| Method | Speed (p50 latency) | Throughput (4-parallel TCP, p50) | Structure | Cost (fixed monthly, always-on) |
|---|---|---|---|---|
| VPC Peering | 8.89ms (fastest) | 10.58Gbps (fastest) | One-to-one only; VPCs growing means a full N(N-1)/2 mesh and unmanageable operations | $0 (only pay-per-GB transfer) |
| Transit Gateway | 11.01ms (+2.1ms vs. Peering) | 9.67Gbps (slowest) | Hub-and-spoke manages many VPCs centrally; static routes need upkeep | ~$204/month (4 attachments) |
| PrivateLink | 13.32ms (+4.4ms vs. Peering) | 9.99Gbps (in between) | Exposes individual services one-way, not the whole VPC; no full-mesh reachability needed | ~$38/month |
VPC Peering wins on both latency and throughput, but TGW and PrivateLink swap places between the two — TGW wins on latency, PrivateLink on throughput — so which characteristic matters more to you can change the decision.
Prioritizing latency, connecting a small number of VPCs simply, one-to-one
Go with VPC Peering. No extra components, simplest, cheapest. But full-mesh management cost grows fast as VPCs increase (and overlapping CIDRs between segments are a no-go).
Centrally managing many VPCs and multiple regions (including future growth)
Go with Transit Gateway. Hub-and-spoke centralizes management, but the deciding factor is whether ~$200/month plus static-route upkeep is worth it.
Exposing a specific service one-way, from a single VPC (no full-mesh reachability needed)
Go with PrivateLink. You scope access to the specific service rather than the whole VPC. Cheaper fixed cost than TGW, but adds the most latency versus VPC Peering.
You can't just pick the fastest or cheapest option — it's a balance of VPC count, future extensibility, the exposure unit you want, tolerable latency, and fixed-cost budget.
9. Constraints and things to remember
A summary, for my own future self, since I'll probably trip over these again:
- VPC Peering isn't transitive — no connecting through an intermediate VPC.
- TGW peering attachments don't support route propagation; static routes give precise control at the cost of operational overhead.
- VPC Peering and TGW both target the same peer VPC CIDR, so their routes can't coexist in one route table. If you ever need both in production, split subnets/route tables to keep them apart.
- Cross-region PrivateLink is limited to NLB-based services; no automatic private DNS resolution, so you must handle the endpoint-specific DNS name explicitly.
- An NLB behind PrivateLink can't have a UDP listener without IPv6 source NAT (neither combined TCP_UDP nor standalone UDP works). UDP requires a dualstack NLB plus IPv6 CIDR configuration.
- Measured MTU for cross-region VPC Peering/TGW peering is 8500 (vs. jumbo frame 9001 in-region), raised from 1500 in a March 2025 update.
10. Conclusion
I built and measured all three managed ways of connecting Tokyo and Osaka over a closed network on AWS. On latency, a clean ordering emerged — same AZ → cross-AZ → VPC Peering → TGW → PrivateLink — and breaking PrivateLink's extra latency down into an NLB cost and a cross-region-endpoint-specific cost was the kind of insight you only get from hands-on work.
On throughput,
c6i.large's invisible network burst credits tripped me up repeatedly, and only restarting the instance and measuring each path alone finally produced clean, meaningful numbers — with TGW and PrivateLink swapping places relative to their latency order. That a benchmark's numbers can swing this much based on run duration, order, and instance state is itself one of the biggest takeaways here, alongside a handful of easy-to-miss constraints (the route table conflict, PrivateLink+NLB's lack of UDP support, the 2-AZ cross-region requirement, PrivateLink not forwarding ICMP) that you only discover by actually building it.If latency requirements aren't strict, choose based on manageability (VPC count, future extensibility, exposure unit); if latency is strict, consider VPC Peering first and budget for the overhead if you end up on TGW or PrivateLink. Speed alone shouldn't decide an architecture — VPCs count, extensibility, exposure scope, operational ease, and fixed cost (~$204/month TGW, ~$38/month PrivateLink, $0 Peering) all need to factor in. I hope this is useful to anyone working through their own inter-region connectivity design.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article