AWS Builder Center

My CI Runner Cost Estimate Dropped 67%. Here’s How I Used terraform-aws-github-runner.

I moved test execution to temporary AWS machines while keeping GitHub Actions workflows and checks. I explain the runner setup, compare costs for the same API tests, and show how I handled cleanup, image testing, and S3 artifacts.

cloud engineer, indie hacker, devops
My GitHub Actions bill for a private application had reached $200+ over part of a billing cycle. Budget blocks were also holding up pull request (PR) checks.
I wanted to keep using GitHub Actions and spend less on the machines running the tests.
So I tried  terraform-aws-github-runner , an open-source Terraform module for setting up self-hosted GitHub Actions runners on AWS.
For context, a runner executes the steps in a workflow job. In my setup, each runner used a temporary Amazon EC2 instance, a virtual machine in my AWS account. GitHub still managed the workflows, checks, and job logs.  GitHub's explanation of self-hosted runners 
In one comparison of the same API tests, I estimated $0.270 per run on GitHub-hosted runners and $0.088035 on AWS. That's 67.4% lower. I rounded it to 67% in the title.
I also had to make sure the machines shut down afterward and that new runners started from a tested image.

Where the 67% Came From

I used the same API build and left the test workflow unchanged. Both runs used RSpec, the API's test framework, and reported the same test counts.
Measurement
GitHub-Hosted Runners
AWS Prebuilt Runners
RSpec examples (test cases)
8,983
8,983
Failures
0
0
Pending examples
4
4
Time spent running RSpec
9m35s
9m28s
Estimated variable runner cost, USD
$0.270
$0.088035
For that workload, the estimated difference was $0.181965 per run, or about 18.2 cents.
For AWS, I counted the machines, their disks, and public IPv4 addresses. I used the recorded startup and cleanup observations to estimate how long the machines existed.
The times in the table cover only the RSpec command. A runner also takes time to start and shut down, so I included that time in the AWS cost estimate.
The 67% applies to this workload's estimated runner cost. I still need to account for all shared AWS services and maintenance effort before I can give a total monthly CI savings figure.

The Module Deserves the Credit

I used v7.11.0 of  terraform-aws-github-runner . The  github-aws-runners  community maintains the project and credits Philips Labs for starting it.
The module handles notifications from GitHub and starts runners when jobs need them. I kept a copy of that exact version in my infrastructure code. That’s what  vendor/  means in the file layout below.
The project uses the MIT license. If you distribute a copy of its code, preserve the copyright and license notices with the code.  Upstream license 

How a GitHub Job Gets an AWS Runner

The AWS side listens for new jobs, starts machines to run them, and cleans up afterward.

Lambda functions manage the requests to start and clean up runners. EC2 runs the actual job and connects outbound to GitHub for work and results. The cleanup icon groups two functions: scale-down and the watchdog.
Here's how one job gets a machine.
  1. GitHub queues a workflow job. The job's  runs-on  labels identify the kind of runner it needs. The GitHub App sends a  workflow_job  webhook, an HTTP notification, to AWS.
  1. An API Gateway HTTP API receives the notification and calls a Lambda function. That function validates the event and adds a request to an Amazon SQS queue.
  1. Lambda polls SQS through an event source mapping, the connection that delivers queued requests to the scale-up function. The function checks that the job is still queued before starting an EC2 runner.  How Lambda reads SQS messages 
  1. The machine boots from a tested runner image. It reads temporary just-in-time (JIT) registration settings from AWS Systems Manager Parameter Store, deletes that temporary parameter, and connects to GitHub for its job.
  1. After the job, GitHub deregisters the runner. The startup script attempts to terminate its EC2 instance. Scheduled cleanup functions provide another way to remove eligible or expired machines if needed.
I used on-demand  m6i.large  and  m7i.large  instances in London, encrypted 30 GiB disks, and a maximum of ten runners per repository. I kept no idle machines ready for new work, often called a warm pool. New jobs could wait for a machine to start.
GitHub calls this an ephemeral runner: it accepts one job and is then deregistered. Deregistration doesn't prove that the EC2 machine has been terminated, so I checked that separately.  GitHub runner reference 


Getting the First Job Running on AWS

To try this in a sandbox, start with the AWS account and network, connect the GitHub App, and deploy the runner infrastructure. The module's getting-started guide lists the full setup requirements. Creating these resources incurs AWS charges.  Module setup guide 

1. Confirm the AWS Account and Region

Start with a short-lived AWS session for a sandbox account. Before reviewing a plan, check the identity and Region your shell will use.
export AWS_PROFILE=your-aws-profile
export AWS_REGION=eu-west-2
aws sts get-caller-identity --profile "$AWS_PROFILE"
Check the returned account and role against the intended environment. If they differ, stop before deploying.
I used a dedicated network with public subnets, public IPv4, and no inbound runner access. The runners required IMDSv2. This design avoided a NAT gateway, while public IPv4 remained part of the cost model.

2. Set Up the GitHub App

Create the GitHub App with the permissions required by the pinned module and subscribe to workflow job events. Install it on the selected private repository.
I stored the App credentials in SSM Parameter Store and passed references to those values into Terraform. The Lambda functions that manage runners could read the App credentials. Each runner received separate temporary JIT settings to register with GitHub.
To let a repository use these runners, include it in both places:
  • The GitHub App installation must include the repository.
  • The infrastructure's repository allowlist must include it too.
After deploying the runner infrastructure, set the App's webhook URL to the endpoint in the deployment output and use its saved webhook secret. GitHub can then send job notifications to the HTTP API.

3. Configure the Runner Module

In  modules/runners/ , my configuration calls the copied module from  ./vendor . This illustrative excerpt shows which repositories can use the runners and how the machines are created. The full configuration also supplies network settings, App parameter references, Lambda release packages, the tested runner image, and a startup template.
module "runners" {
source = "./vendor"

prefix = var.name
aws_region = var.aws_region
vpc_id = var.vpc_id
subnet_ids = var.public_subnet_ids

github_app = {
id_ssm = var.github_app_parameters.id
key_base64_ssm = var.github_app_parameters.key_base64
webhook_secret_ssm = var.github_app_parameters.webhook_secret
}

enable_organization_runners = false
repository_white_list = var.allowed_repositories
runner_extra_labels = ["ci-aws"]

enable_runner_bidirectional_label_match = true
enable_ephemeral_runners = true
enable_jit_config = true
enable_job_queued_check = true

runner_architecture = "x64"
instance_target_capacity_type = "on-demand"
instance_types = ["m6i.large", "m7i.large"]
runners_maximum_count = 10
idle_config = []
pool_config = []
}
The custom label  ci-aws  tells GitHub that a job should use these AWS runners. With bidirectional matching enabled, the job's requested labels must exactly match the runner's labels. That's why the workflow below requests  self-hosted ,  linux ,  x64 , and  ci-aws  together.
 enable_ephemeral_runners  limits each runner to one job.  enable_jit_config  selects the temporary JIT registration method described above. I left  idle_config  and  pool_config  empty so there was no retained idle capacity or scheduled warm pool.
The full configuration also includes encrypted disks, IMDSv2 requirements, and explicit Lambda package paths. Use Lambda packages that match the Terraform module version, and review those inputs before the first apply.

4. Deploy the Infrastructure in Order

I started with Terraform state storage, then the network and runner infrastructure, followed by the image and artifact units. I added image-maintenance automation after I could run a job successfully.
For each configured directory under  deployments/ , I initialized Terraform and saved a plan to review:
terragrunt init
terragrunt plan -out=reviewed.tfplan
Read the plan and verify the target account, resources, permissions, and expected charges. Preserve the working directory and Terragrunt cache until you've applied the saved plan.
Applying that reviewed plan creates or changes AWS resources and can create charges.
terragrunt apply reviewed.tfplan
Once the runner infrastructure points to a tested image, use the webhook endpoint output to finish the GitHub App setup. Check the runner labels and repository allowlist before starting a job.

5. Run a Small Workflow First

I would start with a small job that I can run manually from GitHub Actions. This illustrative workflow belongs in the application's  .github/workflows/  directory.
name: AWS runner smoke

on:
workflow_dispatch:

permissions: {}

jobs:
runner-smoke:
runs-on: [self-hosted, linux, x64, ci-aws]
timeout-minutes: 5
steps:
- name: Check the runner environment
shell: bash
run: |
uname -s
uname -m
docker --version
The four labels match the runner settings above. Read the job log: it should show  Linux ,  x86_64 , and a Docker version. This checks the runner environment before adding application dependencies.
Then move one existing test job to those runner labels. Keep its check name, command, test selection, and result handling unchanged so you can compare the two hosts. Keep a way to switch the job back to GitHub-hosted runners during the migration.

Shutting Down the Runner After the Job

A passing test job was only part of my check. I also needed to confirm that its EC2 instance was terminated afterward.
In the module version I used, the Linux startup script waits for the runner application to finish. When the script exits, its cleanup handler attempts to terminate the instance.
I also had two scheduled cleanup functions: the module's scale-down Lambda and a separate watchdog that checks for runners past their allowed lifetime. I configured the watchdog's limit at 75 minutes. EventBridge rules run both functions on a schedule, independently of the SQS requests that start runners.
I checked the job result, the runner's removal from GitHub, the EC2 instance's final state, and the queue. If EC2 still reports  shutting-down , check again for  terminated  before calling cleanup complete.
That cleanup time belongs in the per-run estimate too. The machine exists for longer than the test command takes to finish.

Testing Images Before New Runners Use Them

Each runner started from a prebuilt Ubuntu 24.04 image with the workload's dependencies and native browser support. On EC2, this is an Amazon Machine Image (AMI): the operating system and software used to boot a new machine.  AWS's AMI guide 
I added automation to build and test new images before using them for normal jobs:
  1. EC2 Image Builder builds a new AMI to test.
  1. A temporary test coordinator starts isolated validation: separate groups of API tests and a small Chromium browser check.
  1. It checks that the test runners were cleaned up and saves versioned test records.
  1. After those checks pass, it writes the new AMI ID to the SSM parameter that selects the image for normal runners. This is the image-promotion step.
  1. A final canary, a small check using the normal runner setup, tests PostgreSQL, Redis, and versioned S3 artifact transfers. The coordinator records the result and terminates.
The latest image test reported 9,659 API examples, zero failures, and three pending. The Chromium check also passed. That run used a different test inventory and execution layout from the earlier cost comparison.
I had to keep Terraform and the image-testing automation from overwriting each other's choice of image. Terraform sets the initial AMI value in Parameter Store. After that, the test automation updates the value, and Terraform leaves those updates alone.
Otherwise, a later Terraform apply could put an old image ID back into use.
The code can switch back to a previous image if the final runner check fails, provided the previous image still meets the configured age rules. I still need to rehearse that rollback with a retained image in the live setup.

Saving Build Outputs to S3

CI jobs sometimes need to pass files to later jobs. I stored these build outputs, or artifacts, in a private S3 bucket with versioning enabled. Downloads request a specific object version and verify its checksum, so a later job can check that it received the intended file.
For access, I used GitHub's OpenID Connect (OIDC) support. A job presents a token that identifies its workflow, and AWS checks that identity against an IAM role's trust policy before issuing temporary credentials. Configure that policy against the actual token claims for the repository and permitted workflow context.  GitHub OIDC documentation 
These permissions let the workflow access its artifacts. The GitHub App credentials used to manage runners serve a separate purpose.
I set CI artifact retention to seven days. Their storage, requests, and any applicable transfer charges also belong in the monthly cost comparison.

The Rest of the AWS Bill

To compare monthly costs, I need the same volume of CI work on both sides and a full account of the AWS costs:
  • Runner machines for their full lifetime, EBS disks, and public IPv4 addresses.
  • API Gateway, Lambda, queues, logs, and the other services that manage runners.
  • Image builds, image tests, and retained images and snapshots.
  • Artifact storage, requests, and applicable data transfer.
I also need to account for the time spent maintaining the setup.
AWS credits helped fund the trial. I excluded them from the cost comparisons. GitHub's plan allowance also affects the bill, so I'd use the allowance and rates that applied to the runs being compared.

Removing the Sandbox Resources

If you reproduce the setup in a sandbox, plan its teardown alongside the deployment.
  1. Pause new dispatches and let active jobs finish. Verify the remaining runner registrations, instances, and queues.
  1. Confirm the AWS identity again and preserve any required evidence, images, artifacts, and state backups.
  1. Review destruction plans for the environment's units in reverse dependency order. Confirm each target before deleting it.
For a configured unit, this first command previews deletion. The second command performs deletion after its interactive confirmation.
terragrunt plan -destroy
terragrunt destroy
Afterward, verify that the setup's resources are gone. Decide which artifact versions and images to preserve before deleting them.

My Take

 terraform-aws-github-runner  let me keep the GitHub Actions workflow the team already used and take control of the machines running it. For the API workload I compared, the same tests ran for an estimated 67.4% lower runner cost.
If you're thinking about doing this, start with one job you know well. Keep the tests unchanged and include the AWS machine's startup and cleanup time when comparing costs. Make sure the instance actually terminates afterward.
You get more control over the runner environment, and you take on its upkeep. I'd use that first job to decide whether the savings justify maintaining the images and cleanup automation before moving more of the pipeline.

References

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article