AWS Builder Center
Running an embedding model inside AWS Lambda when Bedrock access never came through

Running an embedding model inside AWS Lambda when Bedrock access never came through

We built a codebase context engine on AWS in four days. Bedrock model access stayed blocked the whole time, so we bundled a 23 MB ONNX model into the Lambda package instead. Here is what broke and what worked.

We spent four days building Dune, a tool that maps a codebase and tells you where a change belongs, for the First Commit hackathon. Everything runs serverless on AWS in ap-south-1.
The plan was Bedrock for embeddings and generation. That plan died on day one, and the workaround turned out better than the plan. Here is the honest version.

Bedrock never opened

The Model access console page is retired now. It says serverless foundation models enable automatically on first invoke. For Anthropic models there is still a use case form, and submitting it failed:
Your account is not authorized to perform this action.
No explanation of what to fix. An automated support reply pointed at the payment method, since our account is billed through AISPL. We added a valid Indian card, the roughly two rupee verification charge went through, and the error did not change. Titan embeddings were blocked too, so it was not Anthropic specific.
A new account has no way to tell how long this takes or what unblocks it. With a Sunday deadline, waiting was not an option.
One thing worth knowing even if your access works: Claude from India needs the global cross-region inference prefix.
global.anthropic.claude-sonnet-4-6
A plain regional model id fails in a way that does not tell you the prefix is missing.

Putting the model inside the Lambda

We needed embeddings for retrieval. So we put them behind an interface with two implementations, wrote both, and defaulted to the one we could actually run.
interface Embedder {
id: string;
dimension: number;
embed(texts: string[]): Promise<Float32Array[]>;
}
 TitanEmbedder  calls Bedrock at 1,024 dimensions.  LocalEmbedder  runs  all-MiniLM-L6-v2  through transformers.js at 384. Every stored vector set records which embedder and dimension produced it, so a later switch fails loudly instead of silently mixing incompatible vectors.
Getting that to run in Lambda took solving three things.
The model downloading on first use. A cold start that fetches 23 MB is not a cold start anyone wants. The build now downloads the model once, pinned to a Hugging Face revision, and copies it into the deployment package. In Lambda, pointing the library at a local directory with remote downloads disabled means a missing file fails loudly rather than quietly reaching for the network.
onnxruntime needing a Linux arm64 binary. This one solved itself:  onnxruntime-node  already ships a prebuilt linux/arm64 binary inside the npm package. The build copies the package into the artifact and deletes every other platform. Nothing had to be compiled.
sharp's install script being blocked. Text embedding never touches image processing, so sharp gets replaced with a stub at bundle time. The catch: the stub has to be a function, not an empty module. transformers.js checks for sharp as it loads and throws "Unable to load image processing library" if it is falsy. An empty stub would have broken every embedding call.
The result:
Package size
22.6 MB zipped, 45 MB unzipped
Cold start
464 ms init, 1.28 s first call
Warm invocation
8 ms
Peak memory
280 MB of 1,024 MB
Well inside Lambda's 250 MB limit, so no container image was needed. Nothing external sits on the query path.

Lambda gotchas that cost us time

 /tmp  defaults to 512 MB. We clone repositories, and a working tree plus history exceeds that.  EphemeralStorage: 2048  fixes it, but the failure mode before we found it was confusing.
We later replaced  git clone  entirely: the Lambda runtime has no git binary, and GitHub's tarball of a commit contains exactly the tracked files. One download, no git, and faster.
SAM's esbuild builder cannot handle npm workspaces. It copies the package alone and runs  npm install , which 404s on a workspace dependency. We bundle through a Makefile from the repo root instead.  sam build && sam deploy  still works unchanged.
 sam deploy  silently keeps previous parameter values. On an existing stack, a parameter you did not pass keeps its old value. We changed a model id in the template and got "No changes to deploy". Our deploy script now always passes every parameter explicitly.
The browser preflight. Every POST from the browser failed while curl worked fine. API Gateway forwarded the CORS preflight to our Lambda, which answered 404, so the browser blocked the request. curl skips the preflight entirely, which is why no amount of curl testing caught it.

DynamoDB, and the 400 KB wall

Single table, on-demand billing, holding the import graph, code chunks, vectors and the team's saved context. No capacity planning, single-digit millisecond reads, scales to zero between demos.
The item size limit is 400 KB, and both the graph and the vector blobs exceed it on a real repo. We sharded from day one rather than discovering it under load. One detail that would have bitten us: shard keys need zero padding.  GRAPH#10  sorts before  GRAPH#2 , and a reassembled blob in the wrong order is garbage.
Vectors are stored as raw binary, not base64. DynamoDB takes binary directly and base64 would add a third to the size.

What SAM got right

One template, one  sam deploy , and the whole stack came up correctly the first time. API Gateway, four Lambdas, DynamoDB, S3, IAM roles.
That mattered most when we dropped Step Functions. The indexing pipeline was specced as a state machine. Two days in, it was clear that a day of plumbing bought us retries we did not need at demo scale. Moving it to a single Lambda took an afternoon because adding a function and wiring its IAM with a policy template is a few lines.
Amplify Hosting handled the monorepo build once the spec ran  npm ci  at the root, and auto-deploy on push meant the live URL was never stale.
CloudWatch was the only debugging tool we had for deployed functions, and it was enough. Logging the inputs and outputs of every pipeline step from the start is what made that true.

What we would tell ourselves on day one

Deploy a hello-world through the entire stack before writing any feature code. We did, and it found the IAM and region problems on Thursday instead of Saturday night.
Check model access first, then assume it will not come. Interfaces cost an hour and saved the project.
Measure before you reach for the bigger service. We planned OpenSearch Serverless for vector search. Loading the vectors from DynamoDB and computing cosine similarity in the query Lambda takes 220 to 310 ms at our scale, well inside budget. OpenSearch never got built and was never missed.
Dune ended up scoring 29 of 30 on a known-answer evaluation across three repositories, in TypeScript, JavaScript and Python. The eval script is in the repo, so anyone can re-run it.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article