A service returns an error.
The application retries.
It fails again.
The application retries again.
Now imagine thousands of clients doing the same thing at almost the same time.
A small failure can turn into a much larger problem.
This is one reason reliability engineering is not simply about preventing failures. It is also about making sure the system behaves safely when failures happen.
## The Retry Storm Problem
Consider a simple architecture:
User
↓
API
↓
Application
↓
Database
The database temporarily becomes unavailable.
The application sends another request.
It fails again.
Instead of waiting, hundreds or thousands of application instances immediately retry.
The database is already under pressure, but now it receives even more requests.
The result can look like:
Failure
↓
Retry
↓
More traffic
↓
More load
↓
More failures
↓
More retries
The system can amplify its own failure.
## So Should We Stop Retrying?
Not necessarily.
Transient failures can sometimes recover.
A temporary network problem, throttling event, or short service disruption may succeed if the request is attempted again.
The important question is not:
"Should I retry?"
It is:
"How should I retry?"
## Exponential Backoff
Instead of retrying immediately, the client waits progressively longer between attempts.
For example:
Attempt 1 → Immediate
Attempt 2 → Short delay
Attempt 3 → Longer delay
Attempt 4 → Even longer delay
This reduces the pressure placed on the dependency.
AWS recommends controlling retry calls and using exponential backoff when retries are appropriate.
## Why Jitter Matters
There is another problem.
Imagine 10,000 clients receive an error at exactly the same time.
If every client waits exactly 1 second before retrying, they may all send another request at exactly the same time.
That creates another traffic spike.
Jitter introduces randomness into the waiting period.
Instead of:
1 second
1 second
1 second
1 second
clients might retry at slightly different times.
This spreads the traffic instead of creating another synchronized burst.
AWS documentation describes jitter as a way to reduce the "thundering herd" effect caused by synchronized retries.
## Timeouts Are Important Too
A request should not wait forever for another service.
Consider:
Application → Database
If the database does not respond, an unlimited wait can consume application resources while requests continue to accumulate.
A reasonable timeout gives the system a defined boundary.
But an extremely short timeout can also be harmful if it causes unnecessary retries.
The timeout needs to match the behavior and requirements of the workload.
## Not Everything Should Be Retried
This is an important part of the design.
A retry may make sense for a temporary failure.
But blindly retrying every error can create additional problems.
For example, an operation that changes data needs careful consideration before being retried.
This is where idempotency becomes important.
If the same operation is executed multiple times, the system should be designed so that unintended duplicate effects are avoided where appropriate.
## A Safer Request Pattern
A more resilient design can look like:
Request
↓
Timeout
↓
Retry only when appropriate
↓
Exponential Backoff
↓
J
Loading article