AWS Builder Center
Lambda CI/CD with GitHub Actions and S3 reference mode

Lambda CI/CD with GitHub Actions and S3 reference mode

How I automated AWS Lambda deployments with GitHub Actions, self-managed S3 code storage.

Cloud @ Sudo | 14x AWS Certified | 2x Claude Certified | 2x Oracle | 2x CNCF(KCNA , KCSA)
Every Lambda project develops the same habit. Someone needs a quick fix, opens the console, edits the function inline. Nobody commits it. Three weeks later the repo and the deployed function have drifted and nobody's sure which is real.
So I built a CI/CD pipeline in GitHub Actions that deploys the function on every push to  main , and switched the function to Lambda's self-managed S3 code storage so the console editor stops being an option at all.



Where your Lambda code actually lives: COPY mode vs reference mode

When you upload a zip, Lambda doesn't keep it where you put it. It copies the package into its own storage, in a bucket you don't own and can't browse. That's  COPY  mode, the default.
Self-managed S3 code storage points Lambda at a zip in your bucket instead:
aws lambda update-function-code \
--function-name my-function \
--s3-bucket my-artifacts \
--s3-key functions/my-function/build.zip \
--s3-object-version "$S3_VERSION" \
--s3-object-storage-mode REFERENCE
You know it worked when the console tells you the code editor no longer works. That message was the entire reason I did this.

The GitHub Actions pipeline


A developer pushes. GitHub Actions builds a zip, uploads it to S3, points the function at that exact object, and publishes a version tagged with the commit. The workflow authenticates to AWS with OIDC, so no access keys sit in the repo.
That's the whole idea.



The 250 MB Lambda package limit counts layers

A raw install of pandas, numpy, matplotlib and openpyxl came to 246 MB against a 250 MB limit, and attached layers count toward it. Stripping caches, tests and debug symbols got it to 153 MB. Don't strip  *.dist-info : saves 2 MB, and anything calling  importlib.metadata.version()  fails at init without it.
Reference mode takes packages off the 300 GB account quota. The per-package limit is untouched.



Lambda rollback without a rebuild

Every published version records the exact S3 object it came from, including the object version. Leave off  --s3-object-version  and the pointer is just bucket plus key, so overwriting that key retroactively changes what an approved version runs.
The coordinates aren't where you'd guess:
{
"RepositoryType": "S3",
"ResolvedS3Object": {
"S3Bucket": "...", "S3Key": "...", "S3ObjectVersion": "..."
}
}
My first version queried the flat path, got  None None None , wrote  available=true  anyway, and would have called  update-function-code --s3-bucket None  at the exact moment it mattered. A reviewer caught it. I'd tested the happy path and never the failure path, which is the only path a rollback step has.



Three checks worth putting in the pipeline

The storage mode isn't sticky. Omit  --s3-object-storage-mode  on any later call and it quietly reverts to  COPY . The function carries on working, nothing alerts, and the reason you built this evaporates. So the pipeline asserts it after every deploy.
The runtime has to match what you built for. The package targets one Python ABI, so a console runtime change means every deploy afterwards ships binaries it can't load.
And a successful deploy is not a working function.  update-function-code ,  wait  and  publish-version  all succeed for a package that can't import, because the error only shows on invocation. So the build imports the handler module, not just the dependency list:
import main
assert hasattr(main, "handler")
Then the pipeline invokes the published version and fails on an error response.



In reference mode your S3 artifacts are a runtime dependency

This is the part that changed how I think about build artifacts.
In COPY mode an artifact is a record. Lose it, expire it, tidy it away; Lambda has its own copy. In reference mode Lambda periodically re-reads the object to reoptimize the code, so losing access takes the function  Inactive .
The transition isn't immediate. It happens on the next reoptimization pass, so a deleted object looks harmless for a while and then takes the function down at a moment that appears unrelated to whoever cleaned up the bucket. And restoring access alone doesn't fix it; you have to restore access and then run  update-function-code  again.
So: versioning on, and a  Deny  on  s3:DeleteObject  and  s3:DeleteObjectVersion  for the artifact prefix, applied to  Principal: "*" . Admins included. That's the point of it.



Pin the AWS CLI, but mind the minimum version

 --s3-object-storage-mode  is recent and the  ubuntu-latest  image drifts, so I pinned the AWS CLI. The version I picked first was older than the flag, which needs 2.35.12 or later, so the CLI rejected the argument as unknown.





What changed, and what I'd keep

Quota relief was why I looked at reference mode. It isn't why I'd keep it.
Every deployed version traces back to a commit. Rollback is a pointer change. Dependencies live in the same commit as the code using them, so rebuilding in six months gives the same thing.
And nobody edits Lambda code in the console. Not because we agreed not to, but because the editor is switched off and pushing to  main  is easier than the workaround.



Github Action Workflow Try it out:

name: deploy
on:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: read
id-token: write # required for OIDC
# Overlapping runs can interleave: an earlier run failing after a later one
# published would roll back and silently undo the later deploy.
concurrency:
group: deploy-${{ github.workflow }}
cancel-in-progress: false
env:
AWS_REGION: us-east-1
FUNCTION_NAME: my-function
ARTIFACT_BUCKET: my-artifacts-111122223333
PY_VERSION: '3.12'
EXPECTED_ARCH: x86_64
# --s3-object-storage-mode needs CLI >= 2.35.12. Do not pin below that.
AWS_CLI_VERSION: '2.37.1'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PY_VERSION }}
# The runner's preinstalled CLI drifts with the image, so pin it. The
# floor matters: --s3-object-storage-mode was added in 2.35.12 and an
# older pin rejects the argument as unknown.
- name: Install pinned AWS CLI
run: |
curl -sS "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWS_CLI_VERSION}.zip" -o awscliv2.zip
unzip -q awscliv2.zip
sudo ./aws/install --update
aws --version
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
# The package targets one Python ABI. If the function's runtime has
# drifted, every deploy afterwards ships binaries it cannot load - and
# that only surfaces at invocation.
- name: Assert runtime matches the build target
run: |
read -r RUNTIME ARCH <<< "$(aws lambda get-function-configuration \
--function-name "$FUNCTION_NAME" \
--query '[Runtime,Architectures[0]]' --output text)"
echo "function: $RUNTIME / $ARCH"
[ "$RUNTIME" = "python${PY_VERSION}" ] || { echo "::error::runtime is $RUNTIME"; exit 1; }
[ "$ARCH" = "$EXPECTED_ARCH" ] || { echo "::error::arch is $ARCH"; exit 1; }
# Captured before anything changes. In REFERENCE mode the coordinates are
# nested under ResolvedS3Object - there is no flat Code.S3Bucket. Versions
# published before the switch are COPY-mode and expose none at all, so
# guard on the values rather than on $V alone.
- name: Capture rollback target
id: rollback
run: |
V=$(aws lambda list-versions-by-function --function-name "$FUNCTION_NAME" \
--query 'max_by(Versions[?Version!=`$LATEST`], &to_number(Version)).Version' \
--output text 2>/dev/null || echo "None")
[ "$V" = "None" ] && { echo "available=false" >> "$GITHUB_OUTPUT"; exit 0; }
read -r B K O <<< "$(aws lambda get-function --function-name "${FUNCTION_NAME}:${V}" \
--query 'Code.ResolvedS3Object.[S3Bucket,S3Key,S3ObjectVersion]' \
--output text 2>/dev/null || echo "None None None")"
if [ -z "$B" ] || [ "$B" = "None" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
echo "::warning::version $V has no S3 coordinates - rollback unavailable"
exit 0
fi
{ echo "available=true"; echo "version=$V"; echo "bucket=$B"
echo "key=$K"; echo "objver=$O"; } >> "$GITHUB_OUTPUT"
- name: Build
run: |
chmod +x build.sh
PY_VERSION="${{ env.PY_VERSION }}" ./build.sh build/function.zip
- name: Upload and capture object version
id: upload
run: |
KEY="functions/${FUNCTION_NAME}/${GITHUB_SHA}.zip"
S3_VERSION=$(aws s3api put-object \
--bucket "$ARTIFACT_BUCKET" --key "$KEY" \
--body build/function.zip \
--metadata "git-sha=${GITHUB_SHA}" \
--query VersionId --output text)
echo "key=$KEY" >> "$GITHUB_OUTPUT"
echo "s3_version=$S3_VERSION" >> "$GITHUB_OUTPUT"
# --s3-object-storage-mode must be passed EVERY time or it reverts to
# COPY. --s3-object-version is what makes the published version immutable.
- name: Deploy in REFERENCE mode
run: |
aws lambda update-function-code \
--function-name "$FUNCTION_NAME" \
--s3-bucket "$ARTIFACT_BUCKET" \
--s3-key "${{ steps.upload.outputs.key }}" \
--s3-object-version "${{ steps.upload.outputs.s3_version }}" \
--s3-object-storage-mode REFERENCE \
--no-cli-pager
aws lambda wait function-updated-v2 --function-name "$FUNCTION_NAME"
- name: Assert still in REFERENCE mode
run: |
aws lambda get-function --function-name "$FUNCTION_NAME" \
--query Code --output json > code.json
grep -q 'ResolvedS3Object' code.json || {
echo "::error::drifted to COPY"; rm -f code.json; exit 1; }
rm -f code.json
- name: Publish version
id: publish
run: |
V=$(aws lambda publish-version --function-name "$FUNCTION_NAME" \
--description "sha=${GITHUB_SHA:0:7} run=${GITHUB_RUN_NUMBER}" \
--query Version --output text)
echo "version=$V" >> "$GITHUB_OUTPUT"
echo "### Deployed version \`$V\`" >> "$GITHUB_STEP_SUMMARY"
# update-function-code succeeds for a package that cannot import - the
# error only appears on invocation. Without this the pipeline reports
# green on a broken function and the rollback never fires.
- name: Smoke invoke published version
run: |
aws lambda invoke \
--function-name "${FUNCTION_NAME}:${{ steps.publish.outputs.version }}" \
--payload '{"smoke": true}' \
--cli-binary-format raw-in-base64-out \
out.json >/dev/null
cat out.json; echo
python3 - <<'PY'
import json, sys
b = json.load(open("out.json"))
if "errorMessage" in b:
sys.exit(f"::error::{b}")
if not b.get("smoke"):
sys.exit("::error::smoke branch did not run")
print(f"python {b['python']} | pandas {b['pandas']}")
PY
- name: Roll back on failure
if: failure() && steps.rollback.outputs.available == 'true'
run: |
echo "::warning::rolling back to version ${{ steps.rollback.outputs.version }}"
aws lambda update-function-code \
--function-name "$FUNCTION_NAME" \
--s3-bucket "${{ steps.rollback.outputs.bucket }}" \
--s3-key "${{ steps.rollback.outputs.key }}" \
--s3-object-version "${{ steps.rollback.outputs.objver }}" \
--s3-object-storage-mode REFERENCE \
--no-cli-pager
aws lambda wait function-updated-v2 --function-name "$FUNCTION_NAME"



FAQ

What is Lambda self-managed S3 code storage? Lambda normally copies your deployment package into storage it owns. With  S3ObjectStorageMode=REFERENCE , Lambda reads the zip directly from a bucket you control instead. Your packages stop counting against the 300 GB per-region Lambda storage quota, and each published version records the exact S3 object it was built from.
Does reference mode raise the 250 MB Lambda package limit? No. It removes your packages from the 300 GB account quota. The 250 MB unzipped per-package limit is unchanged, and attached layers still count toward it.
Can I still edit Lambda code in the console with reference mode? No. The inline code editor is disabled for functions using reference mode. Changes go through S3 and a function update, which for me was the point rather than a drawback.
Why does my Lambda function go Inactive? In reference mode Lambda periodically re-reads the source object in S3. If it loses access, because the object was deleted or the bucket policy changed, the function transitions to  Inactive . Restoring access alone isn't enough; you also need to run  update-function-code  to force a re-read.
How do I roll back a Lambda deployment? Read the previous published version's S3 coordinates from  Code.ResolvedS3Object  and point the function back at them. No rebuild is needed, because the artifact and the version referencing it both still exist.
Which AWS CLI version supports --s3-object-storage-mode? 2.35.12 or later. Older versions reject the argument as unknown.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article