Retries Are Inevitable: Make AWS Lambda Workloads Idempotent

In distributed systems, a successful operation can sometimes execute more than once. The challenge is not eliminating retries; it is making retries safe.

Kamal Tripathi

Computer Science Student | Cloud Computing & AI Enthusiast

Published Oct 9, 2026

Last modified Oct 9, 2026

The Problem

Consider an AWS Lambda function triggered by Amazon SQS to process payments.

Lambda processes a payment successfully.
The function times out before the message is deleted from the queue.
SQS makes the message available again.
Lambda processes it again, potentially charging the customer twice.

Amazon SQS event source mappings provide at-least-once processing, so duplicate message processing is possible.

The Solution: Idempotency

An operation is idempotent when repeating it with the same logical request does not create additional unintended side effects.

For example, processing the same payment request twice should produce one payment, not two.

AWS Lambda Powertools for Python provides an idempotency utility that can use Amazon DynamoDB to store execution state and previously successful results.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import os

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer,
    IdempotencyConfig,
    idempotent,
)

store = DynamoDBPersistenceLayer(
    table_name=os.environ["IDEMPOTENCY_TABLE"]
)

config = IdempotencyConfig(
    event_key_jmespath="order_id"
)

@idempotent(
    persistence_store=store,
    config=config
)
def lambda_handler(event, context):
    process_order(event["order_id"])
    return {"status": "success"}

How it works: The order_id acts as the idempotency key. When the same request is retried within the configured validity window, Powertools can return the previous successful result instead of executing the function again.

The example assumes process_order() represents the business operation and that the idempotency table and required IAM permissions are configured.

Three Production Tips

Choose the right key: Use a stable business identifier, such as payment_id or order_id, rather than a value that changes between retries.
Handle concurrency: Two identical requests may arrive simultaneously. Use a reliable idempotency mechanism instead of relying on an in-memory flag.
Design for partial failure: Idempotency complements, but does not replace, proper SQS retry policies, dead-letter queues, and partial batch failure handling.

Final Thought

Serverless does not mean failure-free. Retries, timeouts, and duplicate deliveries are normal parts of distributed systems.

Build for retries, not against them. Idempotency is a small design decision that can prevent expensive production bugs.

Builder Center