Building Idempotent Serverless APIs with Amazon API Gateway, Lambda, and DynamoDB
Learn how to make your serverless APIs safe to retry using idempotency keys, DynamoDB conditional writes, and Lambda. This hands-on guide walks through a production-ready payment-processing pattern that prevents duplicate charges and simplifies client error handling.
1
2
3
4
5
6
7
8
9
## Why Idempotency Matters in Serverless
Serverless architectures scale by retrying. Lambda invokes get retried, API Gateway clients time out and resend, and event buses deliver messages more than once. If your API isn't **idempotent**, every retry is a potential duplicate charge, duplicate email, or corrupt record.
An operation is idempotent when calling it multiple times with the same input produces the same result as calling it once.
In this article, I'll show you a pattern I use in production: an idempotency layer built with **API Gateway**, **Lambda**, and **DynamoDB** — no extra infrastructure required.
## The Architecture
Client → API Gateway → Lambda (Idempotency Check)
│
┌─────────┴─────────┐
▼ ▼
DynamoDB Business Logic
(idempotency keys) (process request)
▲
│ store result
└───────────
│
┌─────────┴─────────┐
▼ ▼
DynamoDB Business Logic
(idempotency keys) (process request)
▲
│ store result
└───────────
1
2
3
4
5
6
7
8
9
10
11
12
13
14
## Step 1: The Client Sends an Idempotency Key
The client generates a unique key per *logical* request (a UUID, or a hash of the request payload) and sends it in a header:
```http
POST /payments
Idempotency-Key: 7c9e6679-7425-40de-944b-e07fc1f90ae7
{
"amount": 100,
"currency": "USD",
"orderId": "order-123"
}Step 2: Lambda Checks DynamoDB First
Use a conditional write to atomically claim the key. This is the heart of the pattern:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import json, time, uuid, boto3
from botocore.exceptions import ClientError
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("IdempotencyKeys")
TTL_SECONDS = 24 * 60 * 60 # keep keys for 24 hours
def lambda_handler(event, context):
key = event["headers"].get("idempotency-key") or str(uuid.uuid4())
body = json.loads(event["body"])
now = int(time.time())
try:
# Atomically claim the key — only one request wins
table.put_item(
Item={
"key": key,
"status": "IN_PROGRESS",
"createdAt": now,
"ttl": now + TTL_SECONDS,
},
ConditionExpression="attribute_not_exists(#k)",
ExpressionAttributeNames={"#k": "key"},
)
except ClientError as e:
if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
return handle_existing_key(key)
raise
try:
result = process_payment(body) # your business logic
table.update_item(
Key={"key": key},
UpdateExpression="SET #s = :s, #r = :r",
ExpressionAttributeNames={"#s": "status", "#r": "result"},
ExpressionAttributeValues={":s": "COMPLETED", ":r": json.dumps(result)},
)
return {"statusCode": 200, "body": json.dumps(result)}
except Exception:
# Release the key so the client can retry
table.delete_item(Key={"key": key})
raiseStep 3: Handle Retries Gracefully
If a retry arrives while the original request is still processing, return
409 Conflict. If it arrives after completion, replay the stored result:1
2
3
4
5
6
7
8
9
10
11
def handle_existing_key(key):
item = table.get_item(Key={"key": key}).get("Item")
if item["status"] == "IN_PROGRESS":
return {
"statusCode": 409,
"body": json.dumps({"message": "Request in progress, retry shortly."}),
}
return {
"statusCode": 200,
"body": item["result"], # same response as the original call
}Key Design Decisions
1. TTL on the idempotency table. DynamoDB auto-deletes expired items, so storage costs stay near zero even at high throughput.
2. Store the response, not just a flag. Replaying the original response keeps the client experience consistent — retries are invisible.
3. Delete the key on failure. A failed request shouldn't block legitimate retries. Only successful executions lock the key.
4. Scope keys to the user + operation. In multi-tenant systems, prefix the key (e.g.,
userId#idempotencyKey) to prevent cross-user collisions.Testing the Behavior
| Scenario | First call | Retry (same key) |
|---|---|---|
| Success | 200, processed | 200, cached result |
| In progress | — | 409 |
| Failure | 500, key released | 500, retried fresh |
Use AWS SAM or
sam local invoke to test locally, then hammer the deployed endpoint with a retry loop to verify no duplicates land in DynamoDB.Cost and Performance
This pattern adds one DynamoDB write and one read per request — typically well under $0.01 per million requests on on-demand capacity. Latency impact is single-digit milliseconds. For most APIs, that's a negligible price for guaranteed exactly-once semantics.
Conclusion
Idempotency isn't optional in distributed systems — retries are a feature, not a bug. With a single DynamoDB table and conditional writes, you can make any serverless API safe to retry without queues, streams, or additional services.
The full source code for this example is available in the linked GitHub repository. Happy building!
1
2
Copy the entire block above (including the ` ```markdown ` fences' inner content only) and paste it straight into the **Body** field. Want me to shorten it or swap in a different topic?Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article