One Equation to Size Your Lambda Concurrency: Little's Law
Lambda throttles on concurrency, not requests per second. Learn how Little's Law (L = λW) sizes it from two CloudWatch metrics, verified with a Python simulation.
Mathematics Master's Student | Learning AWS, Data & ML
## Requests per second is the wrong number
Lambda does not throttle you on requests per second. It throttles on **concurrency**: how many requests are in flight at the same moment. By default, an account gets 1,000 concurrent executions per Region, shared by every function in it ([AWS docs]( https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html )).
So the number that decides whether you get `429 TooManyRequestsException` is not "how busy is my API?" It is "how many requests are running right now?"
A single equation from queueing theory answers that. It is called Little's Law, John Little published the proof in 1961, and the Lambda docs use it as their "concurrency formula" without calling it by name. This article explains the math, checks it with a simulation, and covers the three places where it will mislead you if you use it carelessly.
## The math: L = λW
**L = λ × W**
- **L**: the average number of items inside the system. For Lambda, that is concurrency.
- **λ (lambda)**: the average arrival rate. For Lambda, requests per second.
- **W**: the average time one item spends inside the system. For Lambda, the average duration in seconds.
**The everyday version.** Two customers walk into a café every minute, and each stays 15 minutes. On average, 2 × 15 = 30 people are inside. You never counted heads. You got there from the arrival rate and the time each person stays.
**Why it works.** Plot the number of requests in flight over a long window of T seconds. The area under that curve equals the total time all N requests spent running, which is N × W. Divide both sides by T. The average height of the curve is L, and N / T is λ. That gives L = λW. Figure 1, attached at the end, shows this with 12 requests.
**Why engineers like it.** The law holds for any stable system over the long run. It does not care whether traffic arrives in a smooth stream or in bursts, how latency is distributed, or what order requests are served in. You do not need to model your traffic to use it.
**Why it maps cleanly onto Lambda.** A standard Lambda execution environment handles one request at a time and stays busy until that request finishes. So the number of busy environments is the number of requests in flight, which is exactly L.
## Worked examples
Halving your function's duration halves the concurrency it needs. So making the code faster is also capacity planning. The four cases below use the figures from the [Lambda scaling docs]( https://docs.aws.amazon.com/lambda/latest/dg/scaling.html ).
| Traffic (λ, req/s) | Avg duration (W, s) | Concurrency (L = λW) | What it tells you |
| --- | --- | --- | --- |
| 100 | 1.0 | 100 | Concurrency equals RPS only when duration is exactly 1 s |
| 100 | 0.5 | 50 | Same traffic, half the duration, half the environments |
| 200 | 0.25 | 50 | Double the traffic, a quarter of a second each: no change |
| 5,000 | 0.2 | 1,000 | One function uses up the entire default account quota |
The last row is the one to remember. Your account's 1,000 default is shared, so a single busy function at 5,000 req/s and 200 ms would leave nothing for the rest. When that function hits the limit, every other function in the Region gets throttled with it.
## Testing it with a simulation
The law held to within 0.1: the simulation predicted 50.2 and measured 50.1. But the peak was well above the average, and that gap is what actually causes throttling.
The script simulates 10 minutes of traffic at 100 req/s with Poisson arrivals. Each request has a lognormal duration averaging 0.5 s, which gives it the long right tail that real latency has. Every request gets its own environment straight away, so there are no cold starts and no limits. The script samples concurrency every 10 ms.
```python
import numpy as np
RATE = 100 # requests per second (lambda)
MEAN_DURATION = 0.5 # seconds (W)
SIGMA = 0.8 # lognormal shape: higher = longer tail
SIM_SECONDS = 600
rng = np.random.default_rng(42)
# Poisson arrivals
n = rng.poisson(RATE * SIM_SECONDS)
arrivals = np.sort(rng.uniform(0, SIM_SECONDS, n))
# Lognormal durations scaled so the mean is MEAN_DURATION
mu = np.log(MEAN_DURATION) - SIGMA**2 / 2
durations = rng.lognormal(mu, SIGMA, n)
departures = np.sort(arrivals + durations)
# Concurrency on a 10 ms grid, skipping a 10 s warm-up
grid = np.arange(10, SIM_SECONDS - 10, 0.01)
concurrency = (np.searchsorted(arrivals, grid, side="right")
- np.searchsorted(departures, grid, side="right"))
rate = n / SIM_SECONDS
L = rate * durations.mean()
print(f"Little's Law lambda*W : {L:.1f}")
print(f"Using p50 instead : {rate * np.median(durations):.1f}")
print(f"Simulated mean conc. : {concurrency.mean():.1f}")
print(f"Simulated p99 conc. : {np.percentile(concurrency, 99):.0f}")
print(f"Simulated max conc. : {concurrency.max()}")
print(f"Poisson p99 estimate : {L + 2.326 * np.sqrt(L):.1f}")
```
| Measure | Result |
| --- | --- |
| Observed rate (λ) | 100.3 req/s |
| Mean duration (W) / median duration | 0.500 s / 0.359 s |
| Little's Law prediction (λW) | 50.2 |
| Simulated mean concurrency | 50.1 |
| Simulated p99 / max concurrency | 67 / 78 |
| Formula estimate of p99 (L + 2.33√L) | 66.7 |
Figure 2 plots one minute of that run. The line keeps crossing the average of 50 and goes above the p99 line at 67 a few times in that minute.
**Why the peak is predictable.** This is an M/G/∞ system: requests arrive at random and never wait in a queue. In that model, the number in flight at any moment follows a Poisson distribution with mean L, so its standard deviation is √L. The 99th percentile is roughly L + 2.33√L, which gives 66.7. The simulation measured 67.
**What scale does to headroom.** The spread grows with √L, not with L. So larger functions need *proportionally* less headroom:
- At L = 50, p99 sits about 33% above the average.
- At L = 1,000, p99 sits about 7% above it (1,000 + 2.33 × 31.6 ≈ 1,074).
Figure 3 shows the full curve from L = 10 to L = 5,000.
## Three mistakes the formula won't warn you about
1. **Using the median instead of the mean.** Little's Law needs the *average* duration. Latency is right-skewed, so the median sits below the mean. In the simulation, plugging in p50 gave 36 instead of 50, a 28% undercount. In CloudWatch, read `Duration` with the `Average` statistic. Figure 4 shows the gap between the two.
2. **Treating the average as the peak.** A one-minute average hides the bursts inside that minute. Use the busiest minute for λ, then add √L headroom on top. Real traffic is usually burstier than Poisson, so treat L + 2.33√L as the minimum you need, not a safe margin.
3. **Assuming W stays constant.** Duration goes up under load. A cold start adds Init time to the first request on each new environment, and the environment stays busy throughout. A slow database or downstream API raises W, which raises L even when traffic is flat. Separately, each function can add at most 1,000 environments every 10 seconds. Requests that arrive faster than that are throttled with a 429 ([scaling behavior docs]( https://docs.aws.amazon.com/lambda/latest/dg/scaling-behavior.html )).
## A 10-minute sizing checklist
- [ ] **λ:** open the `Invocations` metric with the `Sum` statistic and a 1-minute period. Take the busiest minute and divide by 60.
- [ ] **W:** read `Duration` with the `Average` statistic for that same window, then convert ms to seconds.
- [ ] **L = λW:** check it against `ConcurrentExecutions` with the `Maximum` statistic ([monitoring docs]( https://docs.aws.amazon.com/lambda/latest/dg/monitoring-concurrency.html )). If the two are far apart, your traffic is burstier than the per-minute average suggests.
- [ ] **Headroom:** plan for at least L + 3√L.
- [ ] **Shared quota:** if the result comes close to the 1,000 default, request a quota increase. You can also use reserved concurrency, either to guarantee capacity for critical functions or to cap noisy ones.
- [ ] **Latency-sensitive paths:** set provisioned concurrency from your L, so cold starts don't inflate W ([provisioned concurrency docs]( https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html )).
- [ ] **After every performance fix:** recalculate. A faster function needs fewer environments.
## Takeaway
L = λW tells you the average number of Lambda environments a workload keeps busy. √L tells you how far above that average the peaks will go. Between them, you can size a function from two CloudWatch metrics. You don't need to load-test first or guess.
## Figures
The figures are attached below the article, in this order. All four come from the same simulation as the tables above.
- **Figure 1:** Requests as bars on a timeline, with the count in flight below. The shaded area equals N × W.
- **Figure 2:** One minute of simulated concurrency at 100 req/s. The average (λW = 50.2) is marked against the p99 (67).
- **Figure 3:** How far p99 sits above the average as L grows. It is +33% at L = 50 and +7% at L = 1,000.
- **Figure 4:** The distribution of request durations. The median (0.36 s) gives L = 36, while the mean (0.50 s) gives the correct L = 50.
## Sources
- [Understanding Lambda function scaling]( https://docs.aws.amazon.com/lambda/latest/dg/scaling.html ), AWS Lambda Developer Guide
- [Lambda scaling behavior]( https://docs.aws.amazon.com/lambda/latest/dg/scaling-behavior.html ), AWS Lambda Developer Guide
- [Monitoring concurrency]( https://docs.aws.amazon.com/lambda/latest/dg/monitoring-concurrency.html ), AWS Lambda Developer Guide
- [Configuring provisioned concurrency]( https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html ), AWS Lambda Developer Guide
- J. D. C. Little, "A Proof for the Queuing Formula: L = λW," *Operations Research*, 1961
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article