Why CodeBuild Didn't Work Out for Self-Hosted GitHub Actions Runners at Scale
CodeBuild looked like the obvious managed choice for self-hosted GitHub Actions runners. At scale it did not hold up: slow starts, instance limits, webhook quotas, queue times and pricing.
AWS Container Hero
When we migrated from GitLab to GitHub at Personio , one of the first questions was where our CI jobs should run: on managed runners or on self-hosted ones. CodeBuild looked like the obvious answer. It wasn't. This post explains why we chose it, what went wrong, and what we use now.
Note: This reflects the state as of March 2026 and focuses specifically on CodeBuild for GitHub. Other use cases (e.g., CodePipeline or standalone CodeBuild projects) may behave completely differently.
Why CodeBuild Looked Like the Obvious Choice
My preference was clear: managed runners. Nobody wants to operate CI infrastructure if they don't have to.
GitHub's own hosted runners were not an option for us, for a mix of security, compliance and cost reasons. So the next best thing was something that feels managed but runs in our own AWS accounts. That's exactly what CodeBuild for GitHub Actions promises: connect GitHub, add a label to your workflow, and CodeBuild starts a fresh runner for every job. Apart from a bit of configuration in our AWS accounts, there's nothing to run yourself.
The first tests were promising. The serious issues only showed up during the migration, once real workloads hit it. The worst of them: jobs that were never picked up by a runner at all.
The Requirements
To understand why the issues below hurt, here's what my team needed:
- Immutable runners: exactly one job per runner. No state leaks between jobs, projects or teams.
- Scale: around 1000 parallel builds, with bursts of up to 200 jobs triggered at the same time.
- Fast feedback: jobs should start within seconds, not minutes.
- Compatibility: x86 and ARM, and the usual GitHub Actions ecosystem should just work.
- Reliability: a job that was triggered should get a runner, or we should at least know when it doesn't.
- Reasonable cost per build minute.
I assume you already have a basic understanding of GitHub and AWS. Let's get into what I found.
1. Images Do Not Have the GitHub Runner Pre-Installed
Affects: fast feedback
Problem:
Each CodeBuild instance needs to download the GitHub runner at startup, which costs time.
Workaround:
None. You cannot provide your own AMI for on-demand builds. The runner is installed automatically each time, which adds roughly 10–20 seconds to every build.
Reserved capacity fleets do support custom AMIs. However, fleet instances are reused between builds (AWS even states that cached data on fleet instances "can be accessible to other projects within the same account"), so they do not fit an immutable one-job-per-runner setup.
2. CodeBuild AMIs Are Not Fully Compatible with GitHub Actions
Affects: compatibility
Problem:
- The EC2 images are not fully compatible with GitHub Actions. For example, Node.js is not installed, so JavaScript/TypeScript-based actions fail (see my previous blog post ).
- There is no official Ubuntu ARM image at all, neither as AMI nor as container image. Every ARM option is Amazon Linux.
- GitHub's
hashFileshelper function fails with a runtime error.
Workaround:
None for EC2 mode, as you cannot provide your own AMI (except with reserved capacity fleets, see above).
In container mode, you can bring your own image via the
image:custom-<environment-type>-<image-uri> label (e.g., image:custom-arm-<uri> ).
3. Instance Sizes Are Restricted
Affects: compatibility, cost
Problem:
When using the EC2 option (
linux-ec2 or arm-ec2 ), you can only choose between small, medium, and large instances. That means the maximum is 8 vCPUs and 16 GiB memory.The container-based setup offers more options, but there is a large gap: from 8 to 36 vCPUs on x86 and from 8 to 32 vCPUs on ARM.
Workaround:
Reserved capacity fleets fill the gap with custom instance types or attribute-based compute selection. But as mentioned above, they do not work with immutable runners.
Otherwise, you either live with limited resources or pay for capacity you do not fully use.
4. Webhook Limits Are Shared Across the Account
Affects: scale
Problem:
There is an internal limit on how many GitHub webhook events can be processed. It is not listed on the CodeBuild quotas page , so you cannot increase it via Service Quotas; you must contact AWS Support.
It is also an account-wide limit. If you configure multiple CodeBuild projects listening to organization-wide events (e.g., separate projects for AMD64 and ARM64), you multiply the number of processed webhook events and hit the limit even sooner.
Workaround:
Ask AWS Support to increase the limit or distribute workloads between multiple AWS accounts.
5. Monitoring Is on You
Affects: reliability
Problem:
CodeBuild sets up a webhook to listen to workflow events. However, nothing tells you whether a webhook event from GitHub was actually delivered successfully. I didn't find a centralized view of this in either AWS or GitHub.
GitHub shows delivery status per webhook, but it is difficult to monitor this automatically.
In practice, this happened regularly. CodeBuild sends errors back via the webhook response (e.g., if your configuration is invalid), where they are rarely noticed. And even if CodeBuild responds with HTTP 200, you cannot be sure a runner was actually started.
Workaround:
Build your own monitoring. For example:
- A cron-based tool that checks webhook deliveries (
GET /orgs/{org}/hooks/{hook_id}/deliveries) and reports failures to your monitoring system
- Optional automated retries (
POST /orgs/{org}/hooks/{hook_id}/deliveries/{delivery_id}/attempts)
6. Per-Minute Billing and No Savings Plans
Affects: cost
Problem:
CodeBuild pricing for EC2 compute is per minute (only Lambda compute is billed per second). Reserved capacity fleets are billed per minute too, with a minimum of 60 minutes per instance. EC2, Fargate and ECS Managed Instances, on the other hand, bill per second.
CodeBuild is also not covered by Savings Plans.
Workaround:
None.
7. No Guarantees on Queue Time
Affects: fast feedback
Problem:
CodeBuild provides no SLA or SLO regarding queue duration, and there is no documentation on expected wait times.
In our tests in eu-central-1 and eu-west-1, both P50 and P90 queue times were above 60 seconds.
Workaround:
AWS recommends reserved capacity fleets, but they are not compatible with immutable runners.
Even if you do not require immutability, fleets come with additional restrictions:
- Limited to a single subnet (therefore a single Availability Zone)
- No automatic failover: you can pick a different fleet per job (
fleet:<name>label), but you have to choose it in the workflow yourself
8. Many Build Options That Behave Differently
Affects: compatibility
Problem:
CodeBuild provides many execution options: EC2, containers, and even Lambda. Their behavior differs significantly, so you need to understand these differences in detail to choose the right setup.
9. Runners Are Not Bound to Their Job
Affects: reliability
Problem:
CodeBuild requires a
runs-on label like codebuild-<project-name>-${{ github.run_id }}-${{ github.run_attempt }} . It starts a runner with exactly these labels, but GitHub assigns a job to any runner whose labels match. The runner is not bound to the job it was started for.
As all jobs in a workflow run share the same
run_id , a job with fewer labels (e.g., without instance-size:medium ) can be picked up by the runner of another job. That job then waits for a runner that never comes. AWS documents this in their troubleshooting guide , but it's easy to miss.
Workaround:
Give each job a unique custom label (e.g.,
build , test ) in addition to the CodeBuild label, as AWS recommends. This works, but every workflow author needs to know about it, and a mistake only shows up as a job stuck in the queue for up to 24 hours.
If a runner fails to start at all (which happens more often than you would like), nothing retries or alerts. That's why monitoring (see #5) is essential.
What This Means
Most of the issues above are annoying, but my team could have lived with them. Slower starts, limited instance sizes or per-minute billing are trade-offs you can weigh against not having to run anything yourself.
What my team could not live with was reliability. Again and again, we saw orphaned jobs: jobs that were triggered but never picked up by a CodeBuild runner. In a single repository that's an annoyance. In a monolithic repository with a merge queue and many commits per day, it is a real problem. One stuck job blocks the queue, every commit behind it waits, and someone has to notice, investigate and re-trigger it. That is frustrating for developers and costs a lot of time.
I chose CodeBuild because I expected it to do the heavy lifting: make sure every job gets a runner, and tell me when it doesn't. Instead, failures end up in webhook responses on the GitHub side, jobs silently wait for a runner that never comes, and we had to build our own monitoring just to notice. At that point, it's no longer the managed experience we were looking for. Many of these issues are also not documented, or only in troubleshooting pages.
So my team did what we originally wanted to avoid: we built our own self-hosted runners. The architecture is deliberately simple, based on Auto Scaling groups and plain EC2 instances (that probably deserves its own post). We now own the operations, but in return we control the images, instance types, start times and monitoring. Today, the setup runs up to 3000 instances in parallel and handles bursts of up to 1000 new runners per minute. For our workload profile, it turned out cheaper per build minute than both CodeBuild and GitHub's hosted runners.
Is CodeBuild a bad choice for everyone? No. If you run a moderate number of builds, can live with a minute of queue time, don't need immutable runners (so fleets are an option), or just want to try self-hosted runners without building anything, it can still be a good fit. But if you need immutable runners at scale, check the points above against your requirements before you commit.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article