Setting a Load Balancer in Front of a Boomi Molecule on AWS
Once your two-node Boomi Molecule is running on EC2, here's how I put an Application Load Balancer in front of it, what broke along the way, and how I checked that both nodes were really sharing the work
Where this picks up
In an earlier post I got a two-node Boomi Molecule running on EC2, with shared storage on EFS and both nodes showing Online in Runtime Management. This post is the next step. The Molecule was up, but clients still had to pick one node's address, and if that node went down, their calls failed even though the other node was fine.
The fix is an Application Load Balancer (ALB). It gives clients one stable address, sends each request to a healthy node, and keeps checking both nodes so a stopped one drops out of rotation on its own.
I chose an ALB and not a Network Load Balancer because all the traffic here is HTTP. An NLB is for other protocols or very high throughput, and this didn't need either.
This setup was a personal proof of concept on a Boomi training environment, built to learn how Molecules work.
Account IDs, IPs, hostnames, and resource IDs are replaced with placeholders. Replace each one with the value from your own AWS and Boomi consoles as you go. The table below shows where each value comes from. The example values only show the format. They are not real.
| Placeholder | Where to find it | Example format |
|---|---|---|
<VPC_ID> | VPC console, or the EC2 console under Network | vpc-xxxxxxxxxxxxxxxxx |
<NODE_1_PRIVATE_IP>, <NODE_2_PRIVATE_IP> | EC2 console, each instance, Private IPv4 address | 10.0.x.x or 172.31.x.x |
<NODE_SG_ID> | EC2 console, Security Groups, the group your nodes use | sg-xxxxxxxxxxxxxxxxx |
<ALB_SG_NAME> | The name you give the ALB security group | demo-alb-sg |
<ALB_DNS_NAME> | EC2 console, Load Balancers, DNS name on the Details tab | <name>-xxxxxxxxx.<region>.elb.amazonaws.com |
<ENVIRONMENT_NAME> | Boomi platform, Manage, Runtime Management | Your test environment name |
<NODE_1_NAME> | The name you gave the first EC2 instance | node-1 |
<YOUR_PUBLIC_IP> | Run curl -s https://checkip.amazonaws.com on the machine you test from | x.x.x.x (add /32 for the security group rule) |
Keep these private: installer tokens, your
.pem key file, AWS access keys, account IDs, and your public IP if you don't want it visible. Don't paste them into a forum post, a chat, or a screenshot. If a token is ever exposed, generate a new one.What you need before starting
The Molecule has to be finished. Both nodes should show Online in Cluster Status, one should be the head node, and there should be no Cluster Issues tab. Both nodes should be in one security group, and that group should already allow node-to-node traffic and NFS from itself. You also need SSH access to both nodes.
Two decisions to make first. The Shared Web Server in this setup uses plain HTTP with Authentication Type set to None, which is only acceptable if nobody else can reach it. So I restricted the ALB's security group to my own public IP and never opened it to the internet. The second is cost, which I cover later, because the ALB charge keeps running even when the EC2 instances are stopped.
Here is what the setup looked like:
| Item | Value |
|---|---|
| Region / VPC | us-east-1 / <VPC_ID> (default VPC) |
| Node 1 | <NODE_1_NAME>, private IP <NODE_1_PRIVATE_IP> |
| Node 2 | <NODE_2_NAME>, private IP <NODE_2_PRIVATE_IP> |
| Node Availability Zone | us-east-1b (both nodes) |
| Node security group | <NODE_SG_ID> |
| Molecule environment | <ENVIRONMENT_NAME> |
Public IPs change every time you stop and start an instance, so I don't write them down. I copy them from the EC2 console before each SSH session. Private IPs stay the same.
Part 1: Check the Shared Web Server
Open Runtime Management, go to the Molecule, and open Shared Web Server. Make sure the breadcrumb says the Molecule and not a single Atom, because that's an easy thing to get wrong.
The settings that matter here are API Type set to Advanced, port 9090, SSL unchecked, and Authentication Type set to None. The Base URL column only shows one hostname, which will be node 1's internal name. Every node listens on 9090, and the load balancer talks to each node by its private IP. That internal hostname only resolves inside AWS, so don't try to use it from your laptop.
API Type is a runtime-wide setting, and it decides how a deployed listener process turns into a URL. On Basic or Intermediate, a listener deploys with a fixed URL under
/ws/simple/. On Advanced, listeners are published through an API Service Component, which builds a URL tree under /ws/rest/. This is the detail that caught me out in the first attempt. A listener process deployed by itself on an Advanced runtime deploys with no error, but the /ws/simple/ path doesn't route, and every request returns a 404. The process has to be wrapped in an API Service Component, and both must be deployed. That's why Part 2 exists.A port change only takes effect after a restart. Save first, then restart, then wait about 90 seconds before testing. On a cluster, restart one node at a time: stop node 2, restart node 1, wait until it's green and stable for a minute, then start node 2 again.
Once the nodes are back, check each one directly on the node itself:
1
2
sudo ss -tlnp | grep 9090
curl -i http://localhost:9090/_admin/statusYou want a LISTEN line for java on 9090, and a
200 OK with Content-Length: 0. The empty body is correct. /_admin/status is the health probe the load balancer will call every 15 seconds, and it only needs to return 200.Use
curl -i when you test a probe. The first time I ran it without -i, I got no output and assumed it had failed. The status code was 200 the whole time, and that's the only thing the ALB looks at.Part 2: Build a test API
The load balancer needs something to send traffic to. I built a small test API that echoes the request and reports which node handled it. That way I could see the load balancer alternating between nodes without writing any extra scripts.
The listener process is called
Molecule_GetLoadTest_Listener, and it has four steps. The first is a Web Services Server listener with an EXECUTE operation, object name GetLoadTest, single-document input and output, and text/plain as the response content type. The second is a Set Properties step that copies three values from Execution Property into dynamic document properties: Node Id, Atom Name, and Execution Id. The third is a Message step with the text Echo: {1} | Handled by node: {2} | Molecule: {3} | Execution: {4}. The fourth is Return Documents set to none. For a listener, I turned on "Allow simultaneous executions" and kept the workload at general.Node Id is useful because the Molecule fills it in on clusters. It's the node's private IP with the dots turned into underscores, so you'll see something like
172_31_32_71 in the reply. That's the value that makes the load balancing visible.Next, create an API Service Component. Give it any name, set the Base URL path to
v1, and add a REST route that points at the listener process with the HTTP method set to POST. The effective URL is built from the base path and the object name, so the path comes out as /ws/rest/v1/GetLoadTest. The REST tab on the deployed API shows the final path, so check it there rather than trusting my arithmetic. API Management has to be enabled on your Boomi account for this component type to show up.Deploy both pieces to the Molecule's environment, and make sure you deploy them separately. If you deploy only the API component, the deployment reports success and the routes get registered, but nothing is listening behind them, so requests still fail. If you change the process later, you have to redeploy it as well. Wait about 15 seconds after the last deployment before you test.
To confirm the API is loaded on a node, run:
1
2
curl http://<NODE_1_PRIVATE_IP>:9090/ws/rest/v1/openapi.json
curl http://<NODE_2_PRIVATE_IP>:9090/ws/rest/v1/openapi.jsonPart 3: Test each node before adding the load balancer
I always test the nodes directly before I put a load balancer in front of them. If something is broken, it's much easier to find it now.
I SSH'd into node 2 with its public IP. First, a quick note on the key file. SSH refused to use it with a message about permissions being too open:
1
2
WARNING: UNPROTECTED PRIVATE KEY FILE!
Permissions 0644 for '<KEY_FILE>.pem' are too open.The message makes it sound like a login problem, but it's just the file permissions. Run
chmod 400 on the key, then run the same SSH command again.From node 2, I ran both tests. Node 2 can reach node 1's private IP, so one session covers both nodes:
1
2
3
4
curl -i -X POST -H "Content-Type: text/plain" -d 'hello' \
http://<NODE_1_PRIVATE_IP>:9090/ws/rest/v1/GetLoadTest
curl -i -X POST -H "Content-Type: text/plain" -d 'hello' \
http://<NODE_2_PRIVATE_IP>:9090/ws/rest/v1/GetLoadTestEach reply should be a
200 OK with an echo that names the node that answered. The reply has no trailing newline, so the second response gets glued onto the end of the first in the terminal. That's cosmetic.Don't run these from your laptop. The private IPs only exist inside the AWS network, so from a laptop the connection gets reset. It's the same mistake as trying to SSH with a private IP.
If you get a 404 here, check that both the listener and the API Service Component are deployed to the Molecule's environment. If the connection is refused, the port isn't open, so go back to Part 1.
Part 4: Set up the security groups
There are two firewalls to set up. One belongs to the load balancer, and one rule on the nodes lets the load balancer in.
For the ALB's own security group, I went to EC2, then Security Groups, then Create security group. I named it
<ALB_SG_NAME>, set the description, and picked the same default VPC as the nodes. For inbound, I added one rule: HTTP on port 80, with the source set to My IP. Starting with My IP is safer than "allow everyone," since the web server has no authentication yet. I left the default outbound rule alone. Removing it stops the ALB from sending traffic to the nodes.One gotcha with My IP: it stores your address as a /32 at the moment you choose it. If your public IP changes, for example when you switch networks or turn a VPN on or off, requests will just time out. Edit the rule and pick My IP again.
Then the node side. Open the node security group, click Edit inbound rules, and add a Custom TCP rule on port 9090. For the source, don't type the group name in the Description column, because that column is just a free-text label. Use the Source search box and pick the ALB security group. It shows up as a blue tag with its
sg- ID. Keep the rule from My IP that you use for direct testing.This rule is the one people forget. The ALB sends traffic and health checks to the nodes on 9090 from inside the VPC, and without this rule those are dropped. Nothing in the console tells you clearly what's wrong. Both targets just stay unhealthy, and requests through the ALB return 503.
Part 5: Create the target group
The target group is the list of nodes the ALB can use, plus the health check that decides who's currently in rotation. In EC2, go to Target Groups and create one.
For the basics, set the target type to Instances, the protocol and port to HTTP and 9090, the protocol version to HTTP1, and the VPC to the default VPC so it matches the nodes.
For the health check, use HTTP with the path
/_admin/status, the traffic port, and a success code of 200. Set the healthy threshold to 5, the unhealthy threshold to 2, the timeout to 5 seconds, and the interval to 15 seconds. With those numbers, a node that stops answering gets removed after about 30 seconds (two checks at 15 seconds each), and a node that comes back has to pass five checks in a row before it's trusted again, which takes around 75 seconds.On the register step, select both nodes, check that the port shows 9090, and click Include as pending below. That button is easy to skip. Until you click it, the selection is only a shopping basket, and the group gets created with no targets at all. The review page should list both instances. Note their Availability Zone, because you'll need it when you create the load balancer. Both nodes were in
us-east-1b.The targets will say Unused until a load balancer points at this group. That's normal.
Part 6: Create the load balancer
In EC2, go to Load Balancers, choose Create load balancer, and pick Application Load Balancer.
Name it, make it internet-facing (it's still limited to your IP by the security group), pick IPv4, and choose the default VPC. For Availability Zones, select at least two, and make sure one of them is the zone your nodes are in. An ALB only sends traffic to targets in the zones it's enabled in. If the nodes are in
us-east-1b and you leave that zone out, the targets show as unused or unhealthy and nothing reaches them.For security groups, remove the default one the wizard pre-selects and choose the ALB group. The default group has no rule for your IP on port 80, so the ALB would be created and then be unreachable. For the listener, use HTTP on port 80 forwarding to your target group. I didn't add HTTPS, since there's no certificate in this setup.
On the review page I got a yellow warning that said the load balancer's reachability might be impacted, because the security groups didn't have the required inbound and outbound rules. I checked both tabs in the security group, and the rules were there. The warning had been calculated before I saved the rules, so it was out of date. I refreshed the security group selector and it cleared. If it doesn't clear for you, create the ALB anyway, since you can always edit the security group later without rebuilding the ALB. The warning is only harmless if you've actually checked the rules, though. If the inbound rule is missing, or it's on the wrong port, or the outbound rule was deleted, the same warning is real. Spend 30 seconds checking before you dismiss it.
Click Create. It goes into Provisioning and becomes Active after a few minutes. The DNS name on the Details tab is the address clients will use. Use it as
http://<ALB_DNS_NAME>/ with no port, since the listener is on 80. AWS generates that name, and it changes every time you delete and recreate the load balancer.I left the default attributes alone. Two of them are worth knowing about. Cross-zone load balancing is on, so requests spread across both nodes even though only one zone has nodes. Deletion protection is off, so you can delete the ALB from the console when you want to pause it.
Part 7: Check that it works
Wait for both targets to turn healthy
In Target Groups, open the target group and look at the Targets tab. Both nodes start as Initial, then should become Healthy in about two minutes. Use the refresh icon rather than reloading the whole page.
If a node stays unhealthy, the status text in its row tells you why. In my experience the usual causes are these. The security group rule from Part 4 is missing or has the wrong source. The node's own probe is failing, which you can check with
curl -i http://localhost:9090/_admin/status on that node. Or the Molecule isn't running on that node, which you can check with ./atom status.Test through the load balancer from inside AWS
This test shows the ALB works without any help from a laptop or corporate network. My first attempt timed out:
1
curl: (28) Failed to connect to <ALB_DNS_NAME>:80 after 134836 msThat's a long time to wait. I had added a temporary rule to the ALB group with the source set to the node group, reasoning that node 2 was "in that group." It doesn't work that way. An internet-facing ALB's DNS name resolves to public IPs, so the node calls it over its public address, and a security group source only matches traffic from private addresses. The ALB saw a stranger on port 80 and dropped the packets silently. That's why it looked like a timeout.
The fix is to allow the node's public IP. You can find it by asking the node itself:
1
curl -s https://checkip.amazonaws.comDon't use the public IP from the EC2 console, because it may have changed since you last looked. Then add a new rule with HTTP on port 80, a Custom source, and that address with
/32. Trying to edit the old rule and switch its source to an IP won't work. The console refuses, because a rule's source type can't be changed once it exists. Delete the old rule and add a new one.Always add
--max-time 10 to your curl commands while you're testing. Without it, a blocked request can hang for more than two minutes. With it, a firewall problem fails in 10 seconds, which is much easier to live with.Once the rule was right, the same commands worked:
1
2
3
4
5
6
7
curl -i --max-time 10 http://<ALB_DNS_NAME>/_admin/status
for i in 1 2 3 4 5 6; do
curl -s --max-time 10 -X POST -H "Content-Type: text/plain" -d 'hello' \
http://<ALB_DNS_NAME>/ws/rest/v1/GetLoadTest
echo
doneThe status call returned 200. The six requests were answered by both nodes, and the split wasn't strictly alternating. The ALB picks a healthy target for each request, so that's normal. When you're done testing, delete the temporary rule for your node's public IP.
Test from your laptop
From my Mac, the same request failed with a reset, not a timeout:
1
curl: (56) Recv failure: Connection reset by peerA timeout would mean the security group is blocking you. A reset means something answered and then cut the connection. Running
curl -v and nslookup showed that the hostname resolved to a carrier-grade NAT address, not an AWS address, so security software on my laptop was intercepting the request. The ALB itself was fine, since it had worked from inside AWS. Once I turned the client off for the test, the laptop requests worked.If you hit this, don't just leave the protection off. Ask your IT or security team to allow the ALB hostname. Also remember that the security group needs your current public IP, and if your VPN changes it, the symptom goes back to a timeout.
Confirm the work is actually shared
The replies already show which node handled each request, but the best check is in the platform. Open Process Reporting, find the listener process, and look at the executions. Each request through the ALB makes one execution, and the execution details should show both node IDs. That's the final proof that the cluster is splitting the work.
What I'd still do before using this for real
I haven't tested failover yet. The plan is to stop node 2 and then send the six-request loop again. Every reply should come from node 1 after about 30 seconds. Then I'd start node 2 back up and confirm it returns to the pool after about 75 seconds. This is the test that actually matters, because the whole point is that clients keep working when a node disappears. I'll update this post once I've run it.
Authentication is the other thing. Right now the Shared Web Server has no authentication, so anyone who can reach the ALB can call the API. Before anyone else uses this, switch the authentication type on the port row (Basic is a reasonable start), do a rolling restart, and confirm the health check still returns 200. Without a 200 from
/_admin/status, the nodes will stay unhealthy.HTTPS needs a certificate from AWS Certificate Manager for a name you own. You'd add a 443 listener and open 443 in the ALB's security group. Until then, the listener has to stay on 80.
Keep the ALB security group narrow. My IP is fine for testing, but don't open it to
0.0.0.0/0 while authentication is off.Cost, and how to pause
This is the part that surprised me most. An ALB is billed by the hour for as long as it exists, plus a usage charge, and you can't stop or pause it. Stopping the EC2 instances does nothing to the ALB charge. The target group and security groups are free, so the cheap way to pause is to delete only the ALB and recreate it later with the same settings.
To pause, delete the ALB from its Actions menu, and keep the target group and both security groups. Then stop the Molecule on each node, starting with the member node and finishing with the head node. Run
./atom stop and then ./atom status to confirm it's really down. After that, stop both EC2 instances in the console.To resume, start both instances. Their public IPs will have changed, but the private IPs won't. Check EFS is still mounted (
df -h /mnt/boomi, and run sudo mount -a if it isn't). Start node 1, wait for it to go green and stay stable for a minute, then start node 2. Confirm both nodes show Online in Cluster Status. Then create a new ALB with the same settings, but pick the existing target group and security group instead of making new ones. The DNS name will be different this time, so copy the new one, wait for both targets to turn healthy, and run the tests again.When the experiment is over, delete in this order: the ALB, then the target group, then any leftover security group rules, then the ALB security group, which can't be deleted while the ALB is still using it. After that, follow the teardown steps for the nodes, EFS, and the Molecule.
Mistakes I made, or nearly made
- Private key permissions: SSH refused the key because the file was readable by other users. Run
chmod 400once. - Using an old public IP: Public IPs change when you stop and start instances, so always copy the current one.
- Not restarting after changing the port: Nothing listens on 9090 until you restart. Save, restart, wait 90 seconds.
- Testing a probe without
-i: An empty output looks like a failure, but a 200 with no body is correct. - Building only the listener on an Advanced runtime: Every URL returns 404. Wrap it in an API Service Component and deploy both.
- Deploying only the API component: The deployment says it worked, but requests still fail. Deploy the process too.
- Testing from your laptop with private IPs: The connection gets reset. Test from a node.
- Leaving the default security group on the ALB: The ALB exists but nothing can reach it.
- Not allowing the ALB into the node group on 9090: Both targets stay unhealthy, and the ALB returns 503.
- Typing the group name in the Description column: The rule saves but does nothing. The group goes in the Source search box.
- Skipping "Include as pending below" in the target group: The group gets created with no targets.
- Registering the wrong port: The group has to be on 9090, not 80.
- Leaving out the nodes' Availability Zone: Targets stay unused or unhealthy.
- Ignoring the reachability warning: Check the inbound and outbound rules before you dismiss it.
- Allowing a security group source for an internet-facing ALB: The ALB sees traffic from a public address and drops it. Allow the node's public IP as a
/32instead. - Trying to edit a group-source rule into an IP rule: Delete it and add a new one.
- Running curl without
--max-time: A blocked request hangs for minutes. Always set it. - Assuming an ALB can be paused: It keeps billing. Delete it and recreate it when you need it.
Quick reference
On your laptop,
chmod 400 <key>.pem fixes the key permission problem, and ssh -i <key>.pem ec2-user@<PUBLIC_IP> opens a session on a node. curl -i http://<ALB_DNS_NAME>/_admin/status checks the probe through the ALB, and curl -v --max-time 15 shows exactly which IP you connected to. nslookup <ALB_DNS_NAME> tells you which address the name resolved to.On a node,
curl -i http://localhost:9090/_admin/status checks the local probe, curl -s https://checkip.amazonaws.com prints the node's public IP, and sudo ss -tlnp | grep 9090 shows whether java is listening. From /mnt/boomi/Molecule_<name>/bin, ./atom start, ./atom stop, ./atom restart, and ./atom status control the node.In the console, Target Groups shows health for each node and the reason when one fails, Load Balancers shows the DNS name and listener state, and Security Groups holds the ALB group and the node's 9090 rule.
When something breaks, a 404 usually means the process or API component isn't deployed to this environment, or the URL path is wrong. A 502 or 503 from the ALB means no healthy target, so check the target group status text. A timeout means a firewall dropped the packets. A reset means something in between cut the connection, which is often security software on the laptop. The container logs for each node are under the Molecule's
logs folder, and searching them for sharedServer, jetty, or ws/rest will find most startup problems.Closing
Once I had both targets healthy and the ALB handing requests to both nodes, the setup finally felt like a cluster and not just two machines. The mistakes above were the real time sink. Most of them come down to the same few things: security group sources, public versus private addresses, and skipping a step that doesn't give you any error message. If you hit something different, I'd like to hear about it.
# load-balancer# application-load-balancer# infrastructure# aws-mainframe-modernization# application-integration
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article