AWS Builder Center

AWS Aurora Blue/Green Deployment: Understanding Switchover Downtime

OK

AWS Aurora Blue/Green Deployment: Understanding Switchover Downtime

Introduction

Amazon Aurora Blue/Green Deployments provide a safer way to deploy database changes by maintaining two separate environments: a blue environment that serves production traffic and a green environment where changes can be tested before production switchover.
At first glance, Blue/Green Deployment may appear to provide a zero-downtime database deployment. However, the switchover itself can still introduce a short period of unavailability.
In this article, we will look at how Aurora Blue/Green Deployment works, what happens during switchover, why downtime can occur, and which factors can affect the duration.

What Is Aurora Blue/Green Deployment?

An Aurora Blue/Green Deployment creates a separate green environment based on the existing blue environment.
The architecture can be visualized as:
Application
|
v
ECS / ALB
|
v
Aurora Writer Endpoint
|
+----------+----------+
| |
v |
Blue Environment |
Production DB |
| |
| Replication |
v |
Green Environment <---------+
New DB Version
The blue environment continues serving production traffic while changes are prepared and tested in green.
A typical deployment process is:
  1. Create the green environment.
  1. Aurora replicates changes from blue to green.
  1. Deploy the new application/database version to green.
  1. Run tests and database migrations where supported by the deployment strategy.
  1. Confirm replication is healthy.
  1. Perform the switchover.
  1. Production traffic starts using the green environment.
The important point is that the green environment is not simply another replica inside the same production cluster. Blue/Green Deployment provides a separate environment intended to allow changes to be validated before making them production.

Why Is There Downtime During Switchover?

The most important thing to understand is that Blue/Green Deployment does not mean that the database can switch between environments with absolutely no interruption.
During switchover, Aurora needs to coordinate the transition between the old and new environments.
Conceptually, the process looks like:
Before switchover:

Application
|
v
BLUE
Production traffic
|
v
Database

During switchover:

Application
|
X
|
[Transition]
|
v
GREEN

After switchover:

Application
|
v
GREEN
Production traffic
Existing database connections may be terminated or become invalid during this transition.
Applications then need to establish new connections to the new writer.
Therefore, even when the database is highly available, clients can observe a short interruption.

Multi-AZ Does Not Mean Zero-Downtime Deployment

A common misconception is:
"My Aurora database is Multi-AZ, so switching to another database should have zero downtime."
Multi-AZ and Blue/Green solve different problems.
Multi-AZ is primarily an availability mechanism. It allows Aurora to maintain database availability in the event of infrastructure or instance failures.
Blue/Green Deployment is a deployment mechanism. It allows you to prepare a new database environment and then switch production from blue to green.
For example:
Multi-AZ

Aurora Cluster
|
+---------+---------+
| |
AZ-1 AZ-2
Writer Replica
If the writer fails, Aurora can perform a failover.
Blue/Green is different:
Blue Environment Green Environment
---------------- -----------------
Production New version
| |
+------ Replication -----+
Switchover
|
v

Green becomes production
Having Multi-AZ does not eliminate the transition required between two different environments.

What Happens to Existing Connections?

One of the most important sources of perceived downtime is application connection handling.
Suppose an ECS application currently has:
Application
|
+-- DB Connection 1 --> Blue
+-- DB Connection 2 --> Blue
+-- DB Connection 3 --> Blue
After switchover, those connections may no longer be usable.
The application needs to create connections to the new production database.
This can result in errors such as:
connection reset
connection refused
connection closed
transaction aborted
database connection lost
The exact behavior depends on the application, connection pool, driver, retry logic, and how the database endpoint changes are handled.

Why Can RDS Proxy Help?

RDS Proxy can be useful in architectures where database connection management is important.
Instead of applications connecting directly to Aurora:
ECS
|
+----> Aurora
the architecture becomes:
ECS
|
v
RDS Proxy
|
v
Aurora
The proxy manages database connections and can reduce the impact of database failovers and connection changes.
This does not mean RDS Proxy magically makes Aurora Blue/Green switchover zero-downtime.
However, it can help reduce application-side connection disruption because applications maintain connections to the proxy rather than directly managing connections to the database instances.

Factors That Affect Switchover Downtime

The duration of the interruption is not necessarily constant.
Several factors can influence what users observe.

1. Replication Lag

Before switchover, Aurora needs the green environment to be sufficiently synchronized with blue.
A healthy deployment with very small replication lag is preferable.
For example:
Blue
|
| replication
v
Green

Replication lag: 100 ms
is generally a much better situation than:
Replication lag: 30 seconds
A large replication lag can prevent or delay a safe switchover.

2. Number of Database Connections

An application with many active connections can experience more complicated connection behavior during the transition.
For example:
ECS Task 1
├── 20 DB connections
└── connection pool

ECS Task 2
├── 20 DB connections
└── connection pool

ECS Task 3
├── 20 DB connections
└── connection pool
When the database changes, these connections need to recover or be recreated.

3. Application Connection Pool Configuration

Connection pool settings can have a significant effect on recovery time.
For example:
pool_size = 20
connection_timeout = 10s
idle_timeout = ...
If the application waits too long before detecting a broken connection, users may experience a longer outage.

4. Retry Logic

Applications without database retry logic may immediately return an error when an existing connection becomes invalid.
With appropriate retry behavior:
Request
|
v
DB connection fails
|
v
Retry
|
v
New connection
|
v
Request succeeds
the same database transition can be much less visible to users.
Retries should be implemented carefully, especially around transactions, to avoid accidentally executing non-idempotent operations twice.

5. DNS and Endpoint Resolution

Database endpoints are another important component.
Applications commonly connect using an Aurora endpoint rather than a hardcoded database IP address.
For example:
my-cluster.cluster-xxxx.ap-southeast-1.rds.amazonaws.com
During a switchover, endpoint resolution and connection establishment need to converge on the new environment.
Applications should therefore avoid caching database IP addresses for unnecessarily long periods.

6. Application Startup and Connection Establishment

After the switchover, the application may need to establish new database connections.
If connection creation is slow, the database may already be ready while the application still appears unavailable.
This is why measuring only the database operation is not enough.
A better measurement is an end-to-end request:
Client
|
v
ALB
|
v
ECS
|
v
Database
For example:
POST /login
GET /users/me
If both requests succeed before the switchover and fail during the transition, the observed downtime represents what an actual user experiences.

Measuring the Actual Downtime

A simple monitoring script can repeatedly send requests to the application.
For example:
while true; do
timestamp=$(date '+%Y-%m-%d %H:%M:%S.%3N')

response=$(curl -sk \
-o /dev/null \
-w "%{http_code}" \
https://api.example.com/health)

echo "$timestamp $response"

sleep 0.1
done
You can then trigger the Aurora switchover and observe the period where requests stop succeeding.
For example:
18:00:00.100 200
18:00:00.200 200
18:00:00.300 200
18:00:00.400 503
18:00:00.500 503
18:00:00.600 503
18:00:00.700 200
18:00:00.800 200
In this example, the observed application interruption is approximately:
600 ms
This measurement is more meaningful than simply looking at the RDS event timestamps because it measures the actual client-visible impact.

A Better Production Test

For production-like testing, monitor several layers simultaneously:
Test Client
|
v
ALB
|
v
ECS
|
v
RDS Proxy
|
v
Aurora
Monitor:
  • HTTP status code
  • request latency
  • database connection errors
  • ECS application logs
  • RDS events
  • Aurora replication lag
  • RDS Proxy metrics, if used
  • CloudWatch metrics
This makes it possible to distinguish between:
Database switchover time
and:
Actual application downtime
They are not necessarily the same thing.

Database Migrations Require Special Attention

Blue/Green deployments become more complicated when the new application version includes database schema changes.
For example, suppose version 1 contains:
users
----
id
name
email
and version 2 removes:
email
A migration that immediately drops  email  can create compatibility problems if the blue application is still running and expects that column.
A safer approach is often an expand-and-contract migration.

Step 1 — Expand

Add the new schema without removing the old one.
users
----
id
name
email
phone

Step 2 — Deploy

Deploy application code that can work with both schemas.

Step 3 — Migrate Data

Backfill or transform data as required.

Step 4 — Switchover

Switch production traffic to the new application/database environment.

Step 5 — Contract

Only after the old application is no longer required should obsolete columns or tables be removed.
This approach reduces compatibility problems during deployment.

Recommended Architecture

For an ECS application using Aurora Blue/Green, a robust architecture can look like:
Internet
|
v
ALB
|
v
ECS Service
|
v
RDS Proxy
|
v
Aurora Blue/Green
/ \
BLUE GREEN
Production New Version
The deployment process becomes:
1. Create Green
|
v
2. Replication
|
v
3. Deploy application
|
v
4. Run compatibility tests
|
v
5. Verify replication
|
v
6. Prepare application for reconnect
|
v
7. Switchover
|
v
8. Verify application
|
v
9. Monitor

Key Takeaways

Aurora Blue/Green Deployment significantly reduces the risk of database deployments, but it should not be interpreted as a guarantee of zero client-visible downtime.
The most important points are:
  • Blue/Green provides a separate environment for testing and deployment.
  • Switchover is still a transition between database environments.
  • Existing database connections can be interrupted.
  • Multi-AZ improves database availability but does not eliminate Blue/Green switchover interruption.
  • Replication lag can affect the readiness of the green environment.
  • Connection pools and retry logic strongly influence application recovery time.
  • RDS Proxy can help with connection management but does not guarantee zero downtime.
  • End-to-end monitoring is the best way to measure real user-visible downtime.
  • Expand-and-contract migrations are safer for applications that need backward-compatible schema changes.
In practice, the goal should not simply be to ask whether Aurora Blue/Green has "zero downtime."
A better question is:
How quickly can the application recover from the database connection transition, and how much of that transition is visible to users?
With proper connection management, retry handling, monitoring, and a carefully designed deployment strategy, the impact of the switchover can be reduced significantly.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article