Your Lambda Ran Twice. Did You Just Delete the Same Data Twice?

Handling duplicate events and partial failures in AWS serverless workflows using DynamoDB conditional writes, idempotency keys, and reliable recovery mechanisms.

Shashank Pandey

Backend-focused Full Stack Developer | AWS Serverless & Cloud | Technical Writer

Published Oct 26, 2026

Last modified Oct 26, 2026

In my previous article, What Happens After You Click Delete Account? I explored the architecture behind delayed account deletion.

But here's another question: What happens when the same deletion task executes twice?

Imagine Amazon EventBridge triggers an AWS Lambda function to delete a user's data after 30 days. A retry or duplicate event could cause the same logical operation to be processed again.

What if the first execution partially succeeds before failing?

Let's explore how to handle these scenarios using AWS.

1. Idempotency Keys: Identifying Duplicate Requests

Assign every deletion request a unique, persistent identifier.

1
2
3
4
5
{
"deletion_request_id": "del_8f31a2",
"user_id": "user_123",
"status": "SCHEDULED"
}

Every retry uses the same deletion_request_id, allowing the system to recognize the same logical operation.

An operation is idempotent when repeating it produces the same intended outcome as executing it once.

However, an identifier alone doesn't prevent duplicate processing. We need to enforce that behavior.

2. DynamoDB Conditional Writes: Preventing Race Conditions

Suppose two Lambda invocations receive the same event simultaneously. Both read SCHEDULED and attempt the deletion.

We can use a DynamoDB conditional write to let only one invocation claim the request:

1
2
Update status to PROCESSING
ONLY IF status = SCHEDULED

DynamoDB evaluates the condition and performs the update atomically. Only one invocation can successfully claim the request through this transition.

The other invocation should inspect the current state rather than blindly repeating the operation.

3. Transaction States: Tracking Progress

Maintain explicit states to understand where each request stands.

State	Purpose
SCHEDULED	Waiting for execution
PROCESSING	Claimed by a worker
COMPLETED	All required steps succeeded
RETRYABLE	Failed but can be retried
FAILED	Requires investigation or intervention

But what happens if Lambda crashes after claiming a request?

A lease or timeout can help identify potentially abandoned operations. Recovery logic can then determine whether to resume processing.

An expired lease alone doesn't prove the previous worker has stopped, so concurrent execution must still be handled safely.

4. The Real Challenge: Partial Failures

Imagine account deletion involves three steps:

Delete database records.
Delete files from Amazon S3.
Clean up identity records.

If the first step succeeds but Lambda crashes before the others, retrying the entire workflow could repeat completed operations.

The solution is to combine idempotent operations with durable progress tracking and recovery.

For example, track the completion of individual steps and resume unfinished work. If an operation succeeds but its status update fails, reconciliation or a safe retry must establish the actual outcome.

For related changes within DynamoDB, TransactWriteItems can provide atomicity. However, it cannot combine DynamoDB changes, S3 deletions, and external API calls into one atomic transaction.

For multi-step workflows, AWS Step Functions can orchestrate operations, retries, and error handling. Each operation must still be designed to handle retries safely.

5. Building a Resilient AWS Workflow

A practical architecture could look like this:

Amazon EventBridge Scheduler: Triggers deletion at the scheduled time.
AWS Lambda: Executes individual deletion operations.
Amazon DynamoDB: Stores idempotency keys, transaction states, and progress.
AWS Step Functions: Orchestrates multi-step workflows and retries.
Amazon CloudWatch: Monitors failures and unfinished requests.
Dead-letter queues: Preserve applicable failed events for investigation and redrive.

Configure retries appropriately for each delivery mechanism, monitor requests stuck in PROCESSING, and implement recovery for operations that fail repeatedly.

Conclusion

Serverless architecture eliminates the need for continuously running infrastructure in many workloads, but it doesn't eliminate distributed systems challenges.

Duplicate events, concurrent invocations, and partial failures still happen.

The key is to combine idempotency keys, DynamoDB conditional writes, explicit transaction states, and reliable recovery mechanisms.

Remember: a unique transaction ID identifies an operation, but it doesn't guarantee that the operation executes safely.

The goal isn't to ensure your Lambda runs only once. It's to ensure that running it again doesn't break your system.

Builder Center