Why SAM CLI cannot build your pnpm workspace, and what to do about it
If you run a pnpm workspace and you are trying to deploy Lambda functions with AWS SAM, you are going to hit a wall that has nothing to do with your code. I hit it twice on two different machines during a hackathon, with two completely different error messages, and neither one mentioned the actual cause. Here is what is happening and how to get past it.
The first wall
Our repository is a pnpm workspace. Six Lambda functions under
services/ , shared types under packages/shared , and every service declares its dependency the way pnpm expects:{ "dependencies": { "@mergelab/shared": "workspace:*" }}That
workspace:* is a pnpm protocol. It tells pnpm to link the local package instead of fetching from a registry.Run
sam build with the default Node.js builder and you get this:Build FailedError: NodejsNpmEsbuildBuilder:NpmInstall - NPM Failed: npm error code EUNSUPPORTEDPROTOCOLnpm error Unsupported URL Type "workspace:": workspace:*The reason is in the builder name.
NodejsNpmEsbuildBuilder copies each function's source into a scratch directory and runs npm install there. Not pnpm. npm has no idea what workspace: means, so it dies before esbuild is ever invoked.This is not a bug so much as a mismatch, but it is a mismatch that will catch an increasing number of people, because pnpm workspaces are now a common way to structure a monorepo. The error also points at npm rather than at the builder's choice of package manager, so your first instinct is to go debug your dependencies, which is a dead end.
The fix: bundle it yourself
The insight is that a Lambda deployment package does not need
node_modules at all if you bundle properly. esbuild can inline your shared package into a single file. The only thing that should stay external is the AWS SDK, because the Node 20 Lambda runtime already provides it.So instead of letting SAM run the install, switch to the makefile build method and do the bundling yourself, from the repository root where pnpm's workspace resolution actually works.
A
Makefile at the repo root, one target per function:build-IngestFunction: npx esbuild services/ingest/src/index.ts --bundle --platform=node \ --target=node20 --format=cjs --outfile="$(ARTIFACTS_DIR)/index.js" \ --external:@aws-sdk/*Two things that cost me time here. Make requires actual tab characters for indentation, not spaces, and it fails cryptically if you get that wrong. And quote
$(ARTIFACTS_DIR) . My user directory has a space in it, and unquoted, esbuild parsed the fragments after the space as extra input files and complained that I needed to use outdir when there are multiple inputs. That error message does not point at the space.Then in
template.yaml , for each function:IngestFunction: Type: AWS::Serverless::Function Properties: CodeUri: ../ Handler: index.handler Runtime: nodejs20.x Architectures: [arm64] Metadata: BuildMethod: makefileNote
CodeUri: ../ pointing at the repository root, because that is where the Makefile lives, and Handler: index.handler because the bundle is a single flat file.You also need esbuild resolvable from the root. If it is only a devDependency of each service,
npx esbuild at the root will fail. Add it to the root package.json and run pnpm install so it hoists.The second wall
With that done,
sam build got further and then failed again. On macOS:Error: CustomMakeBuilder:CopySource - [Errno 63] File name too long:/var/folders/.../node_modules/.pnpm/node_modules/@mergelab/daemon/node_modules/@mergelab/hooks/node_modules/@mergelab/daemon/node_modules/@mergelab/hooks/...On Windows, the same underlying problem surfaced as
WinError 3 , a path length failure, with completely different text.Here is what is going on. Because
CodeUri now points at the repository root, SAM copies the entire tree to a temporary build directory. That includes node_modules . Our workspace has a circular dependency between two packages, daemon and hooks , which import each other. In pnpm's symlink based store, that cycle becomes an infinitely nesting path, and the copy walks it until the operating system refuses.The fix is one flag:
sam build --build-in-sourceThis builds in place rather than copying to a scratch directory, so the symlink cycle is never traversed. It worked immediately on both platforms.
I found this flag by reading
sam build --help , not by finding it in documentation about this error, which is the part worth complaining about. Two different operating systems, two unrelated looking errors, one flag.What I would tell past me
Three things.
Check the builder, not your dependencies. When
sam build fails on install, ask which package manager the builder is invoking before you go looking at your own package.json . The builder name tells you.Bundle everything except the AWS SDK. Once you accept that the Lambda artifact should be a single self contained file, a whole category of dependency resolution problems disappears. Verify it too, rather than trusting that the build succeeded:
node -e "require('./.aws-sam/build/IngestFunction/index.js')"If that throws a module not found error, your bundle is incomplete and you will only discover it at runtime in production.
A circular workspace dependency is a real problem, not a tooling quirk. esbuild resolved ours fine because all the cross references are inside function bodies, so it bundled cleanly. But the cycle is what generated that infinite symlink chain, and it will keep breaking tools in unexpected ways. I worked around it for the deadline and wrote it down as debt.
The working setup, in full
# infra/template.yaml, per functionMetadata: BuildMethod: makefileProperties: CodeUri: ../ Handler: index.handler Runtime: nodejs20.x Architectures: [arm64]# Makefile at repo root, tab indentedbuild-IngestFunction: npx esbuild services/ingest/src/index.ts --bundle --platform=node \ --target=node20 --format=cjs --outfile="$(ARTIFACTS_DIR)/index.js" \ --external:@aws-sdk/*cd infrasam build --build-in-sourcesam deploy --guidedSix functions, each a single file of roughly 140KB, no
node_modules shipped, deployed to ap-south-1.If you are running a pnpm workspace and SAM is fighting you, that is where I would start.
#bharatbuilds #BharatBuilds
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article