
Building a FaaS on EC2 Without Lambda — and Splitting Cold Start Into Three Layers
We hosted our own function execution environment on EC2 for a project called NanoGrid. Chasing cold start turned out to be three separate problems stacked on top of each other — container startup, EC2 boot time, and deciding when to scale — and fixing the third one led straight into a security problem we hadn't planned for. Numbers, Terraform, and the ceiling I didn't recognize at the time.
I lead the AWS Student Builder Group at Dongguk University. This article is a retrospective on a project I built before that.
The task was clear: host a serverless execution environment on EC2 ourselves. Upload a function, invoke it over HTTP, have it execute immediately — with no managed FaaS underneath. The platform is called NanoGrid, and I owned infrastructure and security.
But if we were hosting it ourselves anyway, it seemed worth asking whether we could also fix the things that had bothered us about managed FaaS. So before designing anything, I listed Lambda's downsides. At the top of that list was cold start.
This is the story of trying to solve one problem and ending up solving three.

1. The First Fix — Warm Pools
Move cold start onto EC2 and it turns out there are two slow points, sitting at two different time scales.
Containers: seconds
Our first implementation ran
docker run on every request. That means paying for container creation, network namespace setup, and runtime boot each time. The function finishes in milliseconds while preparation takes seconds. The shorter the computation, the more that overhead dominates response time — and short computations are exactly what a FaaS is for.So we kept pre-started container pools per runtime — Python, Node.js, C++, and Go. When a job arrives, nothing is created; a container is borrowed from the pool and returned afterward.
Receive SQS message → acquire container from pool → execute → releaseResult: 3 seconds down to 0.2 seconds. Measured with Prometheus — the worker exposes
worker_job_duration_seconds as a histogram, so the distribution shift before and after was directly visible rather than inferred from a couple of stopwatch runs.Reuse is not free, though, and it is worth being explicit about what you trade for it. A borrowed container is a container that already ran someone else's code, so anything a job leaves behind — files in the working directory, environment mutations, background processes — has to be cleaned up on release or it leaks into the next invocation. This is the same bargain managed FaaS makes on a warm invocation; hosting it yourself just means the cleanup is your code.
EC2 instances: minutes
Fast containers don't help if there's no worker to put them on. A fresh EC2 runs provisioning → boot → user data (installing Docker and packages) → container preparation, from scratch. That's minutes, and it lands on whichever unlucky request triggered the scale-out.
This is where EC2's two states matter: stopped and running. Putting an instance into service really splits into "building it" and "turning it on," and only the first is slow. Finish the building ahead of the traffic, hold the instance stopped, and scale-out becomes nothing but turning it on.
That's an EC2 Auto Scaling Warm Pool.
State | Count | Cost |
|---|---|---|
InService | 2 | Full charge |
Warm pool (stopped) | 3 | No compute charge, EBS only |
Total = ASG max | 5 | |
max capacity − desired capacity = warm pool size (5 − 2 = 3) is AWS's default sizing. You pay for two instances while maintaining readiness for five.We didn't just gain speed here — cost came along with it. Stopped instances carry no compute charge, only EBS volumes. Three 8 GiB gp3 volumes run under $2 a month. Keep those same three running and you pay full instance rates while serving nothing.
aws autoscaling put-warm-pool \ --auto-scaling-group-name nanogrid-workers \ --pool-state Stopped \ --min-size 3 \ --max-group-prepared-capacity 5For transition time AWS writes
as low as 30 seconds . That's a best case, and it covers lifecycle hooks, health checks, and target group registration too — so treat it as the floor, not the number you design against.It worked. A 10,000-request Fibonacci load test finished in about 11 seconds; the same workload on Lambda took 30–40 seconds.
2. Digging Further — Who Does the Autoscaling?
At this point I thought the problem was solved. Then we pushed request volume higher and something else surfaced.
On Lambda, autoscaling is provided by the service itself. You never design scaling at all. We had to build that layer, and a warm pool only supplies prepared instances — it says nothing about when to wake how many.
A conventional web tier watches CPU utilization. But that sequence runs like this:
Traffic spike → CPU climbs → threshold → alarm → scale outThat's late for a FaaS. By the time CPU fills up, the user has already absorbed the delay. CPU is a lagging indicator: it tells you the load arrived, not that it is arriving.
So we moved queuing outside the servers. A request arrives, the controller pushes the job to SQS, and workers pull from the queue. In that shape one SQS message equals one invocation, which makes queue length a leading indicator that mirrors load directly.
Pulling the queue out bought three things:
- Controllers and workers scale independently without knowing about each other
- Request volume is the scaling signal — no separate instrumentation needed
- If a worker dies, the job stays in the queue
resource "aws_cloudwatch_metric_alarm" "sqs_high" { alarm_name = "nanogrid-sqs-high" comparison_operator = "GreaterThanOrEqualToThreshold" threshold = 10 alarm_actions = [aws_autoscaling_policy.scale_out.arn] dimensions = { QueueName = data.aws_sqs_queue.job_queue.name }}Two refinements sit on top, since raw message count invites the objection "plenty of messages, but they drain fast — isn't that over-scaling?"
- Backlog per instance — trigger on
messages ÷ current workers, expanding only past what one worker should reasonably hold
- Step scaling — staged expansion by backlog band (0–10, 10–50) instead of a blanket multiplier
Gotcha — SQS publishes metrics on a one-minute cadence. Set--periodbelow 60 seconds and the data points come back empty, leaving the alarm parked inINSUFFICIENT_DATA. If scale-out never fires at all, start here.
3. The Third Problem — Should We Accept That Volume At All?
With the front of the queue open, another angle appeared. A design that "handles many requests quickly" is, inverted, a design that "accepts many requests as they come."
On top of that, the premise of this platform is running code uploaded by strangers on our infrastructure. We never observed an actual attack, but a public endpoint combined with arbitrary code execution is a condition that warrants getting ahead of.
We put five AWS WAF rule groups in front of the ALB.
Rule | Purpose |
|---|---|
Managed SQLi | SQL injection |
Common Rule Set | XSS, path traversal |
Known Bad Inputs | Log4j and known malicious payloads |
Rate Limit | 1,000 requests per 5 minutes per IP |
Size Constraint | 1 GB body cap (zip bombs) |
The last two mattered most for this project. The rate limit blocks deliberate attempts to flood the queue, and the size constraint stops archives that expand on extraction. Scaling design and security design meet at the same point — without controlling what enters the queue, autoscaling becomes a mechanism for billing you on an attacker's behalf.
The execution layer got constraints in the same spirit: parent-directory traversal blocked on unpack (Zip Slip), per-container CPU and memory quotas, and per-job timeouts. The worker IAM role moved off managed policies like
AmazonSQSFullAccess to a custom policy holding just the three actions it actually needs.
Results
Metric | Value |
|---|---|
Container preparation | 3s → 0.2s |
10,000-request load test | ~11s (Lambda: 30–40s) |
Worker standby cost | 3 stopped instances, under $2/month |
Instance readiness | minutes → tens of seconds |
What I Learned
The biggest takeaway: "cold start" was never one problem.
Layer | What's slow | Time scale | Answer |
|---|---|---|---|
Execution unit | A container per request | Seconds | Docker warm pool |
Worker instance | EC2 boot and initialization | Minutes | EC2 ASG warm pool |
Scaling decision | Deciding too late to expand | Minutes | SQS as a leading indicator |
Fix one and the next layer immediately became the bottleneck. Had we stopped at warm pools, we'd never have seen the autoscaling layer; had we stopped there, the attack surface would have stayed open.
And all three layers stood on one assumption: "the execution unit is a container, and containers run on EC2." Our question was consistently "how do we prepare faster" — but no amount of preparation changes what is being prepared. The approach had a ceiling, and at the time I didn't recognize the ceiling as a ceiling.
Looking back from where I sit now, leading a Student Builder Group, a different axis is visible: "how small can the thing you start actually be?" WebAssembly is that axis. When the execution unit is light enough, there's nothing to pre-warm and no pool to maintain.
What to Look at Next
If you want to work through these layers hands-on, the workshops on AWS Builder Center are a good starting point. The Auto Scaling and container labs are free to run, and you can reproduce this setup in your own account.
And the next article will cover WebAssembly-based FaaS systems. I'm measuring Wasm and container cold starts under identical conditions, taking the numbers myself this time rather than quoting a vendor's.
If you've built something similar — especially if you've ever swapped the execution unit for something other than a container — share it in the comments. Always looking to learn more!
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article