The Same Event Arrived Twice. Should My Application Run It Twice?
I was reading about asynchronous processing and came across a problem that seems simple until you think about the consequences: what happens when the same event gets delivered twice? It made me think about why reliable systems need to handle retries carefully instead of assuming every request or message will be processed only once.
The Same Event Arrived Twice. Should My Application Run It Twice?
Imagine clicking a button to place an order. The page keeps loading, so you click it again.
Now imagine that the first request actually reached the server and the order was already created. The second request arrives a moment later.
From the user's perspective, it was just one order. But the application might end up creating two.
This is the kind of problem that made me look at retries differently.
Retrying a request sounds harmless
When a request fails, retrying it is often the obvious thing to do.
Maybe the network connection dropped. Maybe a service was temporarily unavailable. Maybe the response took too long to arrive.
Trying again makes sense in these situations.
But there is an important detail: a failed response does not always mean the operation failed.
The server might have completed the operation, but the response never reached the client.
If the client retries without checking what happened before, the same operation could happen twice.
This gets more interesting with AWS
Consider an application that sends messages to Amazon SQS and uses AWS Lambda to process them.
SQS helps separate the part of the application that produces work from the part that processes it. The consumer can process messages independently, and the queue can help handle temporary failures.
However, SQS Standard queues provide at-least-once delivery. A message can be delivered more than once.
For example, a Lambda function might finish processing a message but fail before the message is successfully acknowledged. The message can become available again.
The next attempt may process it again.
That does not mean SQS is broken. It means the application needs to account for how distributed systems behave.
The idea of idempotency
This is where idempotency becomes important.
An operation is idempotent when repeating the same operation does not change the intended outcome beyond the first successful execution.
Think about setting a user's account status to
ACTIVE.Setting it to
ACTIVE once or several times produces the same final state.Now compare that with adding ₹500 to an account balance.
If the same request is processed twice, the balance could increase by ₹1,000 instead of ₹500.
The difference is not simply whether an operation can be repeated. It is whether repeating it produces an incorrect result.
How could an application prevent duplicates?
One approach is to give each logical operation a unique identifier.
When a request arrives, the application checks whether that operation has already been completed.
If it has, the application can return the stored result instead of performing the operation again.
If it hasn't, the application processes it and records the outcome.
In a distributed application, that check needs to be designed carefully. Two requests could arrive at nearly the same time, so a separate read followed by a write may not be sufficient. A database constraint or an atomic conditional write can help prevent both requests from claiming the same operation.
For an AWS application, DynamoDB conditional writes can be one way to implement this kind of protection.
The exact design depends on the operation. Sending an email, updating a database row and charging a payment method do not all have the same failure modes.
Retries are useful, but they aren't the whole solution
I used to think of retries mainly as a way to make an application more reliable.
Now I see that retries also create a responsibility: the application has to know what happens when the same work comes around again.
This matters when using queues, serverless functions, external APIs and even ordinary HTTP endpoints.
A retry policy can help recover from temporary failures, but it cannot automatically guarantee that a business operation happens only once.
The application still needs to protect its data and side effects.
What I took away
The interesting part of this problem is that the original request can be completely valid, the infrastructure can be working as expected, and the application can still produce the wrong result.
That is why I think idempotency is worth understanding early when learning backend and cloud engineering.
It changes the question from "How do I retry this operation?" to something more important:
"If this operation runs again, will my application still produce the correct result?"
That is a question I would want to ask before building any workflow where duplicate processing could have real consequences.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article