Dependency rot, CUDA bloat, and Lambda's 10-second init cap: a container image revisited
A container-image Lambda that worked in 2025 stopped building in 2026. Measuring it turned up 2.6 GB of unusable CUDA libraries, an init phase timing out against a 10-second cap, and an image 1.6 GB from a hard service limit
Originally published on Medium .
In May 2025 I built llm-lambda : a
sentence-transformer packaged as a container image and deployed to AWS Lambda with
Terraform. No inference API, no endpoint to keep warm, no GPU. Send it text, get back a
vector, pay only for what you use.
bge-small-en
sentence-transformer packaged as a container image and deployed to AWS Lambda with
Terraform. No inference API, no endpoint to keep warm, no GPU. Send it text, get back a
vector, pay only for what you use.
It worked. I wrote it up, pushed it, moved on.
Going back a year later, three things turned out to be true. The image no longer builds at
all. The two factors most responsible for its cold-start behaviour were both in plain sight
on the day I shipped: one a hard limit in the docs, the other 2.6 GB of GPU drivers I was
shipping into a function that has never seen a GPU, which had quietly pushed the image to
within 1.6 GB of a service ceiling I didn't know existed. And the feature everyone would
tell you to reach for was never available to this design in the first place.
all. The two factors most responsible for its cold-start behaviour were both in plain sight
on the day I shipped: one a hard limit in the docs, the other 2.6 GB of GPU drivers I was
shipping into a function that has never seen a GPU, which had quietly pushed the image to
within 1.6 GB of a service ceiling I didn't know existed. And the feature everyone would
tell you to reach for was never available to this design in the first place.
None of this is a defect in Lambda. It is what happens when you ship something that works
and stop asking why it works.
and stop asking why it works.
What the thing actually is
The whole function is fifteen lines:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from sentence_transformers import SentenceTransformer
import json
model = SentenceTransformer("/var/task/model/bge-small-en")
def lambda_handler(event, context):
text = event.get("text", "")
if not text:
return {"statusCode": 400, "body": "Text missing"}
embedding = model.encode(text).tolist()
return {
"statusCode": 200,
"body": json.dumps({"embedding": embedding[:5]})
}The model is baked into the image rather than pulled at runtime, which is the right call.
Downloading weights from Hugging Face on every cold start would be worse in every way. The
Terraform gives it 2 GB of memory and a 60-second timeout:
Downloading weights from Hugging Face on every cold start would be worse in every way. The
Terraform gives it 2 GB of memory and a 60-second timeout:
1
2
3
4
5
6
resource "aws_lambda_function" "llm_lambda" {
package_type = "Image"
image_uri = var.image_uri
timeout = 60
memory_size = 2048
}Note line 4 of the handler.
handler, which is the standard advice: do expensive setup once per execution environment so
warm invocations skip it. That advice is correct, and that line is where most of this post
ends up. Hold onto it.
SentenceTransformer(...) runs at module scope, outside the
handler, which is the standard advice: do expensive setup once per execution environment so
warm invocations skip it. That advice is correct, and that line is where most of this post
ends up. Hold onto it.
It doesn't build any more
Before any of the analysis, the first thing I did was rebuild the image. It failed.
Not because I changed anything. The repo has three commits, all from May 2025, and I hadn't
touched it. It failed because
transitive dependency to float:
touched it. It failed because
requirements.txt pins five packages and leaves every
transitive dependency to float:
1
2
3
4
5
sentence-transformers==2.4.0
transformers==4.35.2
huggingface-hub==0.17.3
torch==2.0.1
numpy==1.26.4sentence-transformers depends on scikit-learn without a version bound. Today pip
resolves that to scikit-learn 1.9.0, which no longer publishes a cp311 wheel. Python 3.11
has aged out of its support matrix. So pip falls back to the source tarball, which needs
numpy 2.x as a build dependency, which also has no cp311 wheel, which means building numpy
from source too, inside a Lambda base image that ships no C compiler:
1
2
3
4
5
6
Collecting scikit-learn (from sentence-transformers==2.4.0)
Downloading scikit_learn-1.9.0.tar.gz (7.8 MB)
Installing build dependencies: finished with status 'error'
Collecting numpy<2.5.0,>=2
Downloading numpy-2.4.6.tar.gz (20.7 MB)
../meson.build:1:0: ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang'], ...]I pinned scikit-learn to get past it. The next build got further and then died the same
death on Pillow: another unpinned transitive dependency, another version with no cp311
wheel, another source build reaching for a compiler that isn't there:
death on Pillow: another unpinned transitive dependency, another version with no cp311
wheel, another source build reaching for a compiler that isn't there:
1
2
3
Building wheel for Pillow (pyproject.toml): finished with status 'error'
error: [Errno 2] No such file or directory: 'gcc'
ERROR: Failed building wheel for PillowThat's the part worth sitting with. It isn't one unlucky package. It's every package I
didn't name, drifting independently, and I can't tell you how many more are queued up behind
Pillow because I stopped digging. Repairing this properly means resolving the tree as it
stood in May 2025, and that information doesn't exist anywhere. I never wrote it down.
didn't name, drifting independently, and I can't tell you how many more are queued up behind
Pillow because I stopped digging. Repairing this properly means resolving the tree as it
stood in May 2025, and that information doesn't exist anywhere. I never wrote it down.
Five pins felt like enough at the time. They pinned the packages I had heard of. The
dependencies that broke the build are ones I never typed, pulled in by packages I did, and
they broke not because anyone made a mistake but because a year passed.
dependencies that broke the build are ones I never typed, pulled in by packages I did, and
they broke not because anyone made a mistake but because a year passed.
If you take one operational lesson from this post, it's that a lockfile is not pedantry for
a container you intend to still build later.
tool that produces a real lock. The five-line requirements file is how a working image
becomes an unbuildable one with no commits in between.
a container you intend to still build later.
pip freeze the whole resolved tree, or use a
tool that produces a real lock. The five-line requirements file is how a working image
becomes an unbuildable one with no commits in between.
The 10-second wall I didn't know existed
Lambda's
limited to 10 seconds .
Not 60. Not whatever you set as your timeout. Ten. From the docs:
Init phase extensions, runtime bootstrap, and your module-level code — is
limited to 10 seconds .
Not 60. Not whatever you set as your timeout. Ten. From the docs:
TheInitphase is limited to 10 seconds. If all three tasks do not complete within 10
seconds, Lambda retries theInitphase at the time of the first function invocation
with the configured function timeout.
So if importing torch, importing transformers, and deserializing a model takes longer than
ten seconds, Lambda doesn't fail. It throws the init away and does it again, this time
inside your invocation, billed against your 60-second timeout. Lambda does tell you, in a log
line called
reasons I'll come back to.
ten seconds, Lambda doesn't fail. It throws the init away and does it again, this time
inside your invocation, billed against your 60-second timeout. Lambda does tell you, in a log
line called
INIT_REPORT. The catch is that you have almost certainly never seen it, for
reasons I'll come back to.
I wrote this in my own README a year ago, under Troubleshooting:
❌ "Task timed out after 30.00 seconds": Increasetimeoutandmemory_sizein Terraform.
That's me documenting the symptom and prescribing a workaround without understanding the
mechanism. Raising the timeout gives the retried init more room to finish; it doesn't stop
the double initialization. Raising memory genuinely does help, because Lambda scales
CPU with memory and this init is CPU-bound. But I arrived at 2048 MB by trying numbers until
the errors stopped, and I never wrote down which change actually moved it.
mechanism. Raising the timeout gives the retried init more room to finish; it doesn't stop
the double initialization. Raising memory genuinely does help, because Lambda scales
CPU with memory and this init is CPU-bound. But I arrived at 2048 MB by trying numbers until
the errors stopped, and I never wrote down which change actually moved it.
The fix for a slow module-level model load is not a bigger timeout. It's getting init under
ten seconds.
ten seconds.
What it actually costs
I deployed the rebuilt image to Lambda to find out. 2048 MB, 60-second timeout, the same
config as 2025. This is the entire cold start, verbatim:
config as 2025. This is the entire cold start, verbatim:
1
2
3
4
INIT_REPORT Init Duration: 9996.19 ms Phase: init Status: timeout
START RequestId: d5691e39-9024-414d-bfca-c0285fca49c7 Version: $LATEST
END RequestId: d5691e39-9024-414d-bfca-c0285fca49c7
REPORT Duration: 43023.84 ms Billed Duration: 43024 ms Memory Size: 2048 MB Max Memory Used: 505 MBThere it is, in AWS's own words.
milliseconds under the cap. Lambda threw that initialization away, started again inside the
invocation, and charged me for all of it: 43 seconds billed on a function whose actual
work is 40 milliseconds.
Status: timeout on the init phase at 9996 ms, four
milliseconds under the cap. Lambda threw that initialization away, started again inside the
invocation, and charged me for all of it: 43 seconds billed on a function whose actual
work is 40 milliseconds.
The request succeeded. It returned a correct 384-dimension vector. Nothing errored, nothing
alarmed, and if you were watching a dashboard you'd see one slow invocation. The only thing
that tells you what happened is that
fails, or when SnapStart or provisioned concurrency is enabled. A healthy on-demand function
never logs it. So the one line that explains the problem is a line you have never seen in
your own logs and won't go looking for unless you already suspect it.
alarmed, and if you were watching a dashboard you'd see one slow invocation. The only thing
that tells you what happened is that
INIT_REPORT line, and Lambda emits it only when init
fails, or when SnapStart or provisioned concurrency is enabled. A healthy on-demand function
never logs it. So the one line that explains the problem is a line you have never seen in
your own logs and won't go looking for unless you already suspect it.
Timing the phases individually, this is where the ten seconds go:
1
2
3
4
5
6
import torch 8.50s
import sentence_tf 3.84s
load model 5.19s
-- total init 17.53s
first encode 0.51s
second encode 0.04sNote
the memory it's paying for. But on Lambda, memory is how you buy CPU. Allocation is
proportional, one vCPU arrives at 1,769 MB, so that 2 GB was never about RAM. It was me,
a year ago, discovering that a bigger number made the errors go away, and writing "increase
outrun an init budget I didn't know existed.
Max Memory Used: 505 MB against 2048 MB configured. The function needs a quarter of
the memory it's paying for. But on Lambda, memory is how you buy CPU. Allocation is
proportional, one vCPU arrives at 1,769 MB, so that 2 GB was never about RAM. It was me,
a year ago, discovering that a bigger number made the errors go away, and writing "increase
timeout and memory_size" in the README without understanding that I'd been buying CPU to
outrun an init budget I didn't know existed.
At 43 seconds and 2 GB, each cold start costs about $0.0014. That's not the problem. The
problem is 43 seconds of latency on a function advertised as an on-demand embedding endpoint.
problem is 43 seconds of latency on a function advertised as an on-demand embedding endpoint.
Moving the load out of module scope
So I moved it. Both the model load and the
handler behind a module-level cache. The import alone pulls torch, which costs 8.5 seconds
on its own:
sentence_transformers import go inside the
handler behind a module-level cache. The import alone pulls torch, which costs 8.5 seconds
on its own:
1
2
3
4
5
6
7
8
_model = None
def _get_model():
global _model
if _model is None:
from sentence_transformers import SentenceTransformer
_model = SentenceTransformer("/var/task/model/bge-small-en")
return _modelInit went from timing out at 9996 ms to 146–514 ms, across every run I measured. No more
part is unambiguous.
INIT_REPORT, no more discarded initialization, no more paying for the same work twice. That
part is unambiguous.
The cold-start latency is a messier story, and it took a while to see honestly. Five
consecutive cold starts of the identical image, forced by updating function config:
consecutive cold starts of the identical image, forced by updating function config:
1
5,269 ms 16,490 ms 6,829 ms 5,160 ms 7,809 msMedian about 6.8 seconds, spread 5.2 to 16.5. A three-fold spread on an unchanged image and
unchanged code.
unchanged code.
And the first two cold starts after deploying were 56.7 s and 49.9 s, worse than the version
I was trying to fix. I read that as a regression at the time and was wrong. Lambda
downloads and optimizes a container image on first use, and those numbers are dominated by
work that happens exactly once. It took five more runs before the steady state showed up.
I was trying to fix. I read that as a regression at the time and was wrong. Lambda
downloads and optimizes a container image on first use, and those numbers are dominated by
work that happens exactly once. It took five more runs before the steady state showed up.
Which means the clean before/after I wanted doesn't exist. Every number I have for the
original module-scope version is a first-invocation measurement, taken before I understood
that distinction; I never measured it warm-cached. I can tell you init stopped timing out,
because that is deterministic and reproduced every time. I can tell you a warm-cached cold
start is now roughly 5 to 16 seconds. I can't hand you "43 seconds became 7" as a measured
result, because the two figures aren't the same experiment.
original module-scope version is a first-invocation measurement, taken before I understood
that distinction; I never measured it warm-cached. I can tell you init stopped timing out,
because that is deterministic and reproduced every time. I can tell you a warm-cached cold
start is now roughly 5 to 16 seconds. I can't hand you "43 seconds became 7" as a measured
result, because the two figures aren't the same experiment.
The uncomfortable part is what the steady-state number means. Even at its best, this is
~5 seconds before the first request returns, and the actual embedding is 21 ms. Moving the
load bought correctness and visibility. You stop paying twice, and the cost finally shows up
in
seconds of importing torch and deserializing weights is simply what this design costs on a
2 GB execution environment. If that's unacceptable, no amount of moving code around fixes
it, and the answer is provisioned concurrency, or something that isn't Lambda, or not
self-hosting the model at all.
~5 seconds before the first request returns, and the actual embedding is 21 ms. Moving the
load bought correctness and visibility. You stop paying twice, and the cost finally shows up
in
Duration where you can see it. But it did not make the model load faster. Roughly five
seconds of importing torch and deserializing weights is simply what this design costs on a
2 GB execution environment. If that's unacceptable, no amount of moving code around fixes
it, and the answer is provisioned concurrency, or something that isn't Lambda, or not
self-hosting the model at all.
The 2.6 GB of GPU drivers
This is the part I'm least proud of, because the evidence scrolled past me in the build
output every single time and I never read it.
output every single time and I never read it.
torch==2.0.1 on x86_64 Linux is a 620 MB wheel. I could live with that. What I never looked
at was everything it drags in behind it, which pip prints, verbatim, on every build:
1
2
3
4
5
6
7
8
9
10
nvidia_cudnn_cu11-8.5.0.96 557.1 MB
nvidia_cublas_cu11-11.10.3.66 317.1 MB
nvidia_nccl_cu11-2.14.3 177.1 MB
nvidia_cusparse_cu11-11.7.4.91 173.2 MB
nvidia_cufft_cu11-10.9.0.58 168.4 MB
nvidia_cusolver_cu11-11.4.0.1 102.6 MB
triton-2.0.0 63.3 MB
nvidia_curand_cu11-10.2.10.91 54.6 MB
nvidia_cuda_nvrtc_cu11-11.7.99 21.0 MB
nvidia_cuda_cupti_cu11-11.7.101 11.8 MBThat's 1,647 MB of compressed CUDA runtime on the wire, which unpacks to 2,586 MB inside the
image, on top of the 620 MB torch wheel itself. My function has no GPU. It will never have a
GPU. Lambda does not offer one.
image, on top of the 620 MB torch wheel itself. My function has no GPU. It will never have a
GPU. Lambda does not offer one.
For scale: the model I was actually trying to deploy,
libraries that cannot execute in the environment they were being sent to, and paying to pull
and unpack them on every cold start.
model.safetensors for
bge-small-en, is 133.5 MB. I was shipping roughly twelve times the model's weight in GPU
libraries that cannot execute in the environment they were being sent to, and paying to pull
and unpack them on every cold start.
Two ways out, and they compose.
Install the CPU-only build from PyTorch's own index, which drops the CUDA packages entirely
rather than shipping them unused:
rather than shipping them unused:
1
2
--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.0.1+cpuSame Dockerfile, same base image, one line changed:
| as written | CPU-only torch | |
|---|---|---|
| image size | 8.43 GB | 2.66 GB |
/var/task | 4,733 MB | 1,251 MB |
nvidia/* | 2,586 MB | — |
torch | 1,337 MB | 722 MB |
triton | 185 MB | — |
Changing one line removes 5.77 GB.
Now put that first number next to the one constraint I should have known: Lambda's container
image limit is
10 GB uncompressed .
image limit is
10 GB uncompressed .
My image was at 8.4 GB. I was inside 1.6 GB of a hard service ceiling, on a function that
turns a sentence into 384 floats, and I had no idea, because I never once ran
with a fatter CUDA bundle, the deploy would simply have stopped working and I'd have had no
idea why.
turns a sentence into 384 floats, and I had no idea, because I never once ran
docker images and looked. Had I picked a slightly larger model, or pinned a torch version
with a fatter CUDA bundle, the deploy would simply have stopped working and I'd have had no
idea why.
Two caveats on these figures. They exclude the model, since the weights aren't committed to
the repo, so add 133.5 MB for a real deployment. And this is a repaired build, with
scikit-learn, scipy and Pillow pinned to make it compile at all, so it approximates the 2025 image
rather than reproducing it exactly. The 5.77 GB delta between the two columns is exact,
though, because only the torch line differs between them.
the repo, so add 133.5 MB for a real deployment. And this is a repaired build, with
scikit-learn, scipy and Pillow pinned to make it compile at all, so it approximates the 2025 image
rather than reproducing it exactly. The 5.77 GB delta between the two columns is exact,
though, because only the torch line differs between them.
Then arm64, for less than you'd think
Switch to arm64 too, though for less than the headline suggests. The aarch64 torch wheel is
74 MB against x86's 620 MB, which sounds enormous, but that gap is mostly compression. I
built both and measured the same way:
74 MB against x86's 620 MB, which sounds enormous, but that gap is mostly compression. I
built both and measured the same way:
| x86_64 | arm64 | |
|---|---|---|
/var/task | 1,380 MB | 963 MB |
| torch unpacked | 722 MB | 319 MB |
| image | 2.88 GB | 2.35 GB |
About 400 MB, call it 18%, essentially all of it torch. I'd have guessed higher from the
wheel sizes alone, which is the lesson: compare unpacked bytes, not download sizes.
wheel sizes alone, which is the lesson: compare unpacked bytes, not download sizes.
I deployed both to Lambda at 2048 MB and took six or seven cold starts each, discarding the
first two while the image cache warms:
first two while the image cache warms:
| x86_64 | arm64 | |
|---|---|---|
| median cold start | 6.83 s | 5.78 s |
| range | 5.16 – 16.49 s | 4.34 – 14.24 s |
| init | 146 – 514 ms | 109 – 372 ms |
| peak memory | 759 MB | 740 MB |
arm64 comes out roughly 15% faster at the median, which is about what you'd expect with
400 MB less to page in, and it is meaningfully cheaper: $0.0000133334 per GB-second against
$0.0000166667. Combined, a median cold start goes from about $0.00023 to $0.00015, a third
off.
400 MB less to page in, and it is meaningfully cheaper: $0.0000133334 per GB-second against
$0.0000166667. Combined, a median cold start goes from about $0.00023 to $0.00015, a third
off.
I want to be careful with that 15%, though. The two ranges overlap heavily and I have five
and six samples. What I'm confident saying is that arm64 is not slower and appears somewhat
faster; I would not defend the specific percentage. Worth taking for the price alone, and the
image size and latency come along as a bonus.
and six samples. What I'm confident saying is that arm64 is not slower and appears somewhat
faster; I would not defend the specific percentage. Worth taking for the price alone, and the
image size and latency come along as a bonus.
One thing that did surprise me: the very first invocation after deploy was 28.3 s on arm64
against 56.7 s on x86. A smaller image is faster for Lambda to pull and optimize, which
matters more than you'd think if you deploy often. Every deployment pays it once.
against 56.7 s on x86. A smaller image is faster for Lambda to pull and optimize, which
matters more than you'd think if you deploy often. Every deployment pays it once.
There's a catch worth knowing before you try it.
aarch64, since that index is x86-only, and it doesn't need to, because there's no CUDA on ARM to
strip, so the stock PyPI aarch64 wheel is already CPU-only. If your requirements are properly
pinned, the pin itself is now architecture-specific and needs markers:
torch==2.0.1+cpu does not exist for
aarch64, since that index is x86-only, and it doesn't need to, because there's no CUDA on ARM to
strip, so the stock PyPI aarch64 wheel is already CPU-only. If your requirements are properly
pinned, the pin itself is now architecture-specific and needs markers:
1
2
torch==2.0.1+cpu ; platform_machine == "x86_64"
torch==2.0.1 ; platform_machine == "aarch64"So "just switch to arm64" is not a one-line change once you've locked your dependencies,
which is worth noting given how this post started. The build script also hardcodes
questioned it.
which is worth noting given how this post started. The build script also hardcodes
--platform linux/amd64 for no reason I can defend. It was the default and I never
questioned it.
The standard fix never applied
The usual answer to Lambda cold starts is SnapStart: initialize once when you publish a
version, snapshot the memory and disk state, restore from the snapshot instead of running
init again. For a function whose entire cold-start cost is "load a model into memory," that
is exactly the right shape of solution.
version, snapshot the memory and disk state, restore from the snapshot instead of running
init again. For a function whose entire cold-start cost is "load a model into memory," that
is exactly the right shape of solution.
SnapStart does not support container images.
That's the whole answer. It was true in May 2025, it's true now, and it applies to every
function packaged the way this one is. Python SnapStart also requires 3.12 or later, and
this image is built on
neither of which I noticed.
function packaged the way this one is. Python SnapStart also requires 3.12 or later, and
this image is built on
public.ecr.aws/lambda/python:3.11. Two disqualifications deep,
neither of which I noticed.
The remaining lever was provisioned concurrency, which pre-initializes environments and
bills you for them whether or not anyone calls the function. It works. It also deletes the
reason I chose Lambda in the first place, which was scale-to-zero.
bills you for them whether or not anyone calls the function. It works. It also deletes the
reason I chose Lambda in the first place, which was scale-to-zero.
The pricing asymmetry that quietly disappeared
There used to be a real cost argument against container images that nobody mentioned: the
functions on managed runtimes. Same cold start, different bill, purely because of packaging.
Init phase was billed for container images and custom runtimes, but free for ZIP
functions on managed runtimes. Same cold start, different bill, purely because of packaging.
That ended on 1 August 2025, when AWS
standardized INIT billing
across all packaging types and runtimes.
standardized INIT billing
across all packaging types and runtimes.
Two months after I shipped this, one of the arguments against the design stopped being an
argument, not because containers got cheaper but because everything else got more
expensive. I'll take it.
argument, not because containers got cheaper but because everything else got more
expensive. I'll take it.
What I'd build today
Three things exist now that didn't in May 2025.
Bedrock has made the build/buy math hard to argue with. Titan Text Embeddings V2 is $0.02
per million tokens, handles 8K of context, and emits 256, 512, or 1024 dimensions. Run it
against this function: at 2 GB on x86, a 100 ms warm invocation costs about $0.0000035
including the request charge, so roughly $3.50 per million calls, if every one of them is
warm. A million documents at 100 tokens each through Bedrock is $2.00, with no image to
build, no torch version to pin, and no init phase to lose sleep over.
per million tokens, handles 8K of context, and emits 256, 512, or 1024 dimensions. Run it
against this function: at 2 GB on x86, a 100 ms warm invocation costs about $0.0000035
including the request charge, so roughly $3.50 per million calls, if every one of them is
warm. A million documents at 100 tokens each through Bedrock is $2.00, with no image to
build, no torch version to pin, and no init phase to lose sleep over.
Self-hosting still wins on three counts, and they're the only three that matter: the text
never leaves your function, you control exactly which model version runs, and you aren't
subject to Bedrock's per-region on-demand throughput limits. If none of those apply to you,
you are doing infrastructure work you could have bought for two dollars.
never leaves your function, you control exactly which model version runs, and you aren't
subject to Bedrock's per-region on-demand throughput limits. If none of those apply to you,
you are doing infrastructure work you could have bought for two dollars.
Lambda Managed Instances, announced at re:Invent 2025, targets this project's actual
problem. Functions run on EC2 instances that Lambda provisions and patches, with no cold
starts, multi-concurrent invocations per environment, and EC2 pricing including Savings
Plans. Initialization gets up to 15 minutes instead of 10 seconds. The wall I spent this
whole post describing simply isn't there.
problem. Functions run on EC2 instances that Lambda provisions and patches, with no cold
starts, multi-concurrent invocations per environment, and EC2 pricing including Savings
Plans. Initialization gets up to 15 minutes instead of 10 seconds. The wall I spent this
whole post describing simply isn't there.
I could not determine whether it accepts container images. The API reference treats
interaction between them, which suggests
page lists managed runtimes only, starting at Python 3.13, and every example in the getting-
started guide creates functions with
way from documentation that doesn't say. If you're considering it for a containerized
function, test that before you plan around it.
CapacityProviderConfig and PackageType as independent parameters and documents no
interaction between them, which suggests
Image is allowed. But the user guide's runtime
page lists managed runtimes only, starting at Python 3.13, and every example in the getting-
started guide creates functions with
--package-type Zip. I'm not going to assert either
way from documentation that doesn't say. If you're considering it for a containerized
function, test that before you plan around it.
Either way it comes with a capacity provider, a VPC, and scaling to your configured minimum
rather than to zero, three instances by default for AZ resiliency. It solves the cold start
by removing the cold, which is a good trade for steady traffic and a bad one for a function
called eleven times a day.
rather than to zero, three instances by default for AZ resiliency. It solves the cold start
by removing the cold, which is a good trade for steady traffic and a bad one for a function
called eleven times a day.
Lambda MicroVMs went GA on 22 June 2026: Firecracker-backed, VM-level isolation, up to
eight hours of runtime, built from a Dockerfile you upload to S3 and snapshot. Eight hours
is not what an embedding endpoint needs. But it's the first Lambda-family primitive where
"my container takes a while to warm up and then I want to use it for a long time" is the
intended shape rather than something you fight.
eight hours of runtime, built from a Dockerfile you upload to S3 and snapshot. Eight hours
is not what an embedding endpoint needs. But it's the first Lambda-family primitive where
"my container takes a while to warm up and then I want to use it for a long time" is the
intended shape rather than something you fight.
The version I'd write now
Keep the container, keep the model in the image, change four things.
Lock the dependency tree. Not five pins, the whole resolved graph, so the image that builds
today still builds in 2027. This is the one that would have cost me nothing at the time and
is now the difference between a working artifact and a broken one.
today still builds in 2027. This is the one that would have cost me nothing at the time and
is now the difference between a working artifact and a broken one.
Install CPU-only torch. That single line takes the image from 8.43 GB to 2.66 GB and is by a
wide margin the highest-leverage change available.
wide margin the highest-leverage change available.
Then build for arm64: 400 MB smaller, 20% cheaper per GB-second, and measurably no slower,
with a median cold start of 5.78 s against 6.83 s on x86. Useful and compounding, but much smaller than
the first change; don't let the 74 MB wheel figure fool you into expecting more.
with a median cold start of 5.78 s against 6.83 s on x86. Useful and compounding, but much smaller than
the first change; don't let the 74 MB wheel figure fool you into expecting more.
Move the model load and its imports into the handler behind a module-level cache. This is
the change I'd argue hardest for and the one whose benefit is most easily oversold: it takes
init from a 10-second timeout to ~150 ms and stops you paying for the same work twice, but it
does not make loading faster. Steady-state cold starts still landed between 5 and 16 seconds.
It buys correctness and visibility, not speed.
the change I'd argue hardest for and the one whose benefit is most easily oversold: it takes
init from a 10-second timeout to ~150 ms and stops you paying for the same work twice, but it
does not make loading faster. Steady-state cold starts still landed between 5 and 16 seconds.
It buys correctness and visibility, not speed.
Then measure
image has been pulled at least twice. The first two cold starts of a fresh image include
one-time optimization work and will lie to you about your steady state.
Init Duration in the REPORT line, and measure it more than once, after the
image has been pulled at least twice. The first two cold starts of a fresh image include
one-time optimization work and will lie to you about your steady state.
Drop
itself before your code runs; the execution role governs what the function does, and this
function talks to nothing. Two managed policies where one would do, and the extra one grants
read access to every ECR repository in the account.
AmazonEC2ContainerRegistryReadOnly from the execution role. Lambda pulls the image
itself before your code runs; the execution role governs what the function does, and this
function talks to nothing. Two managed policies where one would do, and the extra one grants
read access to every ECR repository in the account.
The honest summary
The project did what it said, on the day it said it. An embedding model does run in a Lambda
container, the Terraform does provision it, and the cost at low volume is genuinely close to
zero.
container, the Terraform does provision it, and the cost at low volume is genuinely close to
zero.
But I shipped it without understanding the constraint that governs whether the design works,
without checking the size of my largest dependency, and without locking the tree that
holds it together. Then I wrote up the symptom of hitting that constraint as a
troubleshooting tip. The README told readers to raise the timeout, which stops the error appearing and leaves the cause in place.
without checking the size of my largest dependency, and without locking the tree that
holds it together. Then I wrote up the symptom of hitting that constraint as a
troubleshooting tip. The README told readers to raise the timeout, which stops the error appearing and leaves the cause in place.
The unbuildable image is the one that stings, because it's the failure with the longest
fuse and the cheapest fix. Everything else here is a judgment call I'd defend as reasonable
for the time. That one was avoidable, and the bill arrived a year late.
fuse and the cheapest fix. Everything else here is a judgment call I'd defend as reasonable
for the time. That one was avoidable, and the bill arrived a year late.
On the numbers: everything here is measured. The image sizes come from real builds, the
init timings from the function deployed to Lambda at 2048 MB. The one caveat is that the
dependency tree had to be repaired before it would build at all, so this is the 2025 design
on 2026 pins rather than a byte-exact resurrection. The original CloudWatch data is long gone
and I haven't tried to reconstruct it.
init timings from the function deployed to Lambda at 2048 MB. The one caveat is that the
dependency tree had to be repaired before it would build at all, so this is the 2025 design
on 2026 pins rather than a byte-exact resurrection. The original CloudWatch data is long gone
and I haven't tried to reconstruct it.
If you're building this today, use Bedrock unless you have a specific reason not to. If you
have that reason: lock your dependency tree, read what pip is actually downloading, and
measure
have that reason: lock your dependency tree, read what pip is actually downloading, and
measure
Init Duration before you tune anything else.Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article