
Multi-VPC Load Balancing for Amazon EKS workloads
Authors: Ashok Srirama, Sai Charan Teja Gopaluni, Sai Kumar Samala
Introduction
In traditional Amazon Elastic Kubernetes Service (Amazon EKS) deployments, when you create a Kubernetes Service of type LoadBalancer , the AWS Load Balancer Controller provisions a Network Load Balancer (NLB) in the same Amazon VPC as your Amazon EKS cluster. However, certain use cases may require placing your load balancers in a different VPC than the one hosting your Amazon EKS cluster. For example, you might need this separation for enhanced security isolation, where public-facing load balancers reside in a dedicated perimeter network VPC. Another common scenario is centralized networking management, where a dedicated networking team maintains all load balancers in a separate VPC to enforce consistent routing policies and simplify compliance requirements.
This article demonstrates how to configure the AWS Load Balancer Controller to create AWS ELBs in a different VPC (VPC B) than the one hosting your EKS cluster (VPC A). This architecture provides enhanced security isolation, network segmentation, and more flexible network design options.
Use Cases
Why would you want to deploy AWS ALB/NLBs in a separate VPC from your EKS cluster?
- Security Isolation: Separate public-facing components (ALBs) from your application workloads for enhanced security.
- Network Segmentation: Maintain a clear separation between internet-facing infrastructure and internal workloads.
- Compliance Requirements: Meet regulatory requirements that mandate separation of public and private resources.
- Centralized Load Balancer Management: Manage all load balancers in a dedicated VPC across multiple application VPCs.
- Shared Services Model: Create a shared services VPC for common infrastructure components like load balancers.
- Legacy Integration: Connect modern EKS workloads with existing load balancer infrastructure in separate VPCs.
Prerequisites
Before you begin, ensure you have:
- An AWS account with appropriate permissions
- AWS CLI configured with access to your account
- kubectl
- eksctl
- helm
- Two Amazon VPCs:
- VPC A: Where your Amazon EKS cluster is deployed
- VPC B: Where you want to deploy your AWS ALB/NLBs
Implementation Overview
Here's a high-level overview of what we'll accomplish:
- Create VPC A and VPC B (if they don't already exist)
- Deploy an Amazon EKS cluster in VPC A
- Set up VPC peering between VPC A and VPC B, alternatively you can utilize AWS Transit Gateway to connect the VPCs
- Configure route tables and security groups
- Install and configure the AWS Load Balancer Controller with the VPC B ID
- Deploy a sample application with appropriate annotations
- Verify the load balancers created in VPC B

High level architecture
Detailed Implementation Steps
Step 1: Create two VPCs (if they don't exist)
Create two VPCs if they don't exist. We'll use AWS CloudFormation templates for this purpose.
1
export AWS_REGION="us-west-2" # Replace with your AWS regionFor VPC A (EKS Cluster VPC):
1
2
3
4
aws cloudformation create-stack \
--stack-name eks-vpc \
--region $AWS_REGION \
--template-url https://amazon-eks.s3.amazonaws.com/cloudformation/2020-10-29/amazon-eks-vpc-private-subnets.yamlFor VPC B (AWS ELB VPC):
1
2
3
4
5
6
7
aws cloudformation create-stack \
--stack-name elb-vpc \
--region $AWS_REGION \
--template-url https://aws-quickstart.s3.amazonaws.com/quickstart-aws-vpc/templates/aws-vpc.template.yaml \
--parameters ParameterKey=AvailabilityZones,ParameterValue=${AWS_REGION}a\\,${AWS_REGION}b\\,${AWS_REGION}c \
ParameterKey=NumberOfAZs,ParameterValue=3 ParameterKey=PublicSubnetTag1,ParameterValue='kubernetes.io/role/elb=1' \
ParameterKey=CreateNATGateways,ParameterValue=falseStep 2: Create an Amazon EKS Cluster in VPC A
Retrieve VPC A ID
1
2
export VPC_A_ID=$(aws cloudformation describe-stacks --stack-name eks-vpc --query "Stacks[0].Outputs[?OutputKey=='VpcId'].OutputValue" \
--output text --region $AWS_REGION)Retrieve Subnet IDs
1
2
export SUBNET_IDS=$(aws cloudformation describe-stacks --stack-name eks-vpc --query "Stacks[0].Outputs[?OutputKey=='SubnetIds'].OutputValue" \
--output text --region $AWS_REGION)Create EKS Cluster with eksctl:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
cat > eks-cluster.yaml << EOF
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: cross-vpc-demo
region: "${AWS_REGION}"
version: '1.33'
vpc:
subnets:
private:
${AWS_REGION}a: { id: $(echo $SUBNET_IDS | cut -d ',' -f3) }
${AWS_REGION}b: { id: $(echo $SUBNET_IDS | cut -d ',' -f4) }
managedNodeGroups:
- name: ng-1
instanceType: t3.medium
desiredCapacity: 2
privateNetworking: true
EOF
eksctl create cluster -f eks-cluster.yamlStep 3: Set up VPC Peering between VPC A and VPC B
Get VPC B ID
1
2
export VPC_B_ID=$(aws cloudformation describe-stacks --stack-name elb-vpc --query "Stacks[0].Outputs[?OutputKey=='VPCID'].OutputValue" \
--output text --region $AWS_REGION)Create VPC peering connection
1
2
3
4
5
6
export PEERING_CONNECTION_ID=$(aws ec2 create-vpc-peering-connection \
--vpc-id $VPC_A_ID \
--peer-vpc-id $VPC_B_ID \
--query "VpcPeeringConnection.VpcPeeringConnectionId" \
--region $AWS_REGION \
--output text)Accept the peering connection
1
2
aws ec2 accept-vpc-peering-connection --vpc-peering-connection-id $PEERING_CONNECTION_ID --region $AWS_REGION
echo "VPC Peering Connection ID: $PEERING_CONNECTION_ID"Step 4: Update Route Tables
Update route tables in both VPCs to route traffic through the peering connection:
Get CIDR blocks for both VPCs
1
export VPC_A_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_A_ID --query "Vpcs[0].CidrBlock" --output text --region $AWS_REGION)1
export VPC_B_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_B_ID --query "Vpcs[0].CidrBlock" --output text --region $AWS_REGION)1
2
3
4
5
export VPC_A_ROUTE_TABLES=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_A_ID" \
--query "RouteTables[*].RouteTableId" \
--region $AWS_REGION \
--output text)Get route table IDs for VPC B
1
2
3
4
5
export VPC_B_ROUTE_TABLES=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_B_ID" \
--query "RouteTables[*].RouteTableId" \
--region $AWS_REGION \
--output text)Update route tables in VPC A to route to VPC B
1
2
3
4
5
6
7
8
9
10
echo "$VPC_A_ROUTE_TABLES" | tr '\t' '\n' | while read -r RT_ID; do
echo "Creating route for route table: $RT_ID"
aws ec2 create-route \
--route-table-id "$RT_ID" \
--destination-cidr-block "$VPC_B_CIDR" \
--vpc-peering-connection-id "$PEERING_CONNECTION_ID" \
--region "$AWS_REGION"
doneUpdate route tables in VPC B to route to VPC A
1
2
3
4
5
6
7
8
9
10
echo "$VPC_B_ROUTE_TABLES" | tr '\t' '\n' | while read -r RT_ID; do
echo "Creating route for route table: $RT_ID"
aws ec2 create-route \
--route-table-id "$RT_ID" \
--destination-cidr-block "$VPC_A_CIDR" \
--vpc-peering-connection-id "$PEERING_CONNECTION_ID" \
--region "$AWS_REGION"
doneStep 5: Install AWS Load Balancer Controller with VPC B ID
Create pod-identity add-on
1
aws eks create-addon --cluster-name cross-vpc-demo --addon-name eks-pod-identity-agent --region "$AWS_REGIONCreate Service Account:
1
kubectl create serviceaccount aws-load-balancer-controller -n kube-systemPod Identity association for AWS Load Balancer controller
1
eksctl create podidentityassociation --cluster cross-vpc-demo --namespace kube-system --service-account-name aws-load-balancer-controller --role-name aws-load-balancer-controller --well-known-policies awsLoadBalancerController --region us-east-1Install the AWS Load Balancer Controller in your EKS cluster, specifying VPC B as the target VPC
1
2
3
helm repo add eks https://aws.github.io/eks-charts
helm repo update
helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=cross-vpc-demo --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller --set vpcId=$VPC_B_IDImportant: The key configuration here is setting
--set vpcId=$VPC_B_ID, which instructs the controller to create load balancers in VPC B instead of the EKS cluster's VPC.Step 6: Deploy a Sample Application
Get public subnet IDs from VPC B
1
2
3
4
export VPC_B_PUBLIC_SUBNETS=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_B_ID" "Name=map-public-ip-on-launch,Values=true" \
--query "Subnets[*].SubnetId" --region $AWS_REGION \
--output json | tr -d '[]" ' | tr ',' ' ')Create a sample deployment and service
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
cat > sample-app.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: sample-app
spec:
replicas: 2
selector:
matchLabels:
app: sample-app
template:
metadata:
labels:
app: sample-app
spec:
containers:
- name: nginx
image: nginx:1.21
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: sample-app-lb-service
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "external"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: sample-app
EOF
kubectl apply -f sample-app.yaml
Step 6: Verify the NLB is Created in VPC B
Get the service details
1
kubectl get service sample-app-lb-serviceCheck the NLB in AWS console or using AWS CLI
1
2
aws elbv2 describe-load-balancers \
--query "LoadBalancers[?VpcId=='$VPC_B_ID']" --region $AWS_REGIONStep 7: Create AWS ALB in VPC B
Create a ClusterIP service and Ingress
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
cat > sample-app-ingress.yaml << EOF
apiVersion: v1
kind: Service
metadata:
name: sample-app-service
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: sample-app
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sample-app-ingress
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: sample-app-service
port:
number: 80
EOF
kubectl apply -f sample-app-ingress.yaml
Step 8: Verify the ALB is Created in VPC B
Get the service details
1
kubectl get ingress sample-app-ingressCheck the ALB in AWS console or using AWS CLI
1
2
aws elbv2 describe-load-balancers \
--query "LoadBalancers[?VpcId=='$VPC_B_ID']" --region $AWS_REGIONConsiderations
- The LoadBalancer target type must be
ip - The subnets in VPC B must have following tag
kubernetes.io/role/elb: 1for public ELBs andkubernetes.io/role/internal-elb: 1for internal ELBs.
Conclusion
In this article, we've demonstrated how to configure the AWS Load Balancer Controller to create AWS NLB and ALBs in a separate VPC from your EKS cluster. This architecture provides enhanced security isolation, network segmentation, and more flexible network design options.
By separating your public-facing load balancers from your application workloads, you can implement a more secure and compliant infrastructure design while still leveraging the power and flexibility of Amazon EKS and the AWS Load Balancer Controller.
This approach can be extended to more complex scenarios, such as using AWS Transit Gateway for connecting multiple VPCs or implementing more sophisticated network security controls between your load balancers and application workloads.
Additional Resources
- AWS Load Balancer Controller Documentation
- Amazon EKS Documentation
- VPC Peering Guide
- AWS Security Group Documentation
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article