AWS Builder Center
How I automated cross-account DocumentDB dump/restore with Step Functions, ECS Fargate and S3 Replication

How I automated cross-account DocumentDB dump/restore with Step Functions, ECS Fargate and S3 Replication

Serverless pipeline for automated Amazon DocumentDB dump/restore between AWS accounts with cross-region replication, Lambda validation, and Step Functions orchestration. Cost: ~$0.10 per execution.

TL;DR: We built a serverless pipeline that dumps all DocumentDB databases from one AWS account (São Paulo), replicates cross-account/cross-region to another account (Ohio), validates file integrity, and restores automatically — all without human intervention. Cost per execution is ~$0.10 and the full flow takes ~1-2 hours.



1. The Problem

Our team maintains an Amazon DocumentDB cluster in one environment (sa-east-1) that needs to be periodically replicated to another environment in a different region (us-east-2, in a separate AWS account). The scenario:
  • Environment A (source): DocumentDB cluster with multiple databases in  sa-east-1  (São Paulo)
  • Environment B (destination): DocumentDB cluster in  us-east-2  (Ohio), separate account
  • Frequency: On-demand (with the option to schedule via cron)
  • Volume: Variable-sized databases, ~5-20 GB total

What we did before

The process was 100% manual:
  1. Someone connected via bastion host in the source VPC
  1. Ran  mongodump  manually for each database
  1. Compressed and uploaded to S3
  1. Downloaded in the destination environment
  1. Ran  mongorestore  on each database
Problems:
  • Took ~2 hours of an engineer's manual work
  • Risk of human error (forgetting a database, wrong credentials)
  • No consistency guarantee across databases
  • No visibility into the process (nobody knew if it worked until manually checking)



2. DocumentDB-Specific Challenges

Amazon DocumentDB is "MongoDB compatible" — but it's not MongoDB. This brings important challenges:

Mandatory TLS

DocumentDB requires TLS/SSL on all connections. You need to download the  global-bundle.pem  certificate from AWS before any operation:
wget https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
And use these flags on every connection:
--ssl --sslCAFile /tmp/global-bundle.pem --sslAllowInvalidHostnames

Private Network (VPC-only)

DocumentDB is only accessible from within the VPC. This means any automation must run inside the VPC — you can't do it from a local machine or a Lambda outside the network.

Unsupported Indexes

Not all MongoDB index types work on DocumentDB. During  mongorestore , you'll see errors like  "Unsupported feature" . Documents are restored normally — only some advanced indexes fail. We need to treat this as a warning, not a fatal error.

retryWrites=false

DocumentDB doesn't support  retryWrites . The connection string must include  ?retryWrites=false , otherwise the connection fails silently.



3. Solution Architecture

Serverless pipeline architecture for cross-account DocumentDB dump and restore using Step Functions, ECS Fargate, S3 Replication, EventBridge and Lambda


Components

Component
Purpose
Why this choice
ECS Fargate
Runs mongodump/mongorestore
Runs inside the VPC, no server to maintain, pay only when running
Step Functions
Orchestrates the flow
Automatic retry, timeout, visibility, logs
S3 + Replication
Stores dumps and replicates cross-account
AWS-native, cross-region, no code needed
EventBridge
Automatic trigger
Detects marker file in S3, starts restore
Lambda
Validates replication
Ensures ALL files arrived before restore starts
Secrets Manager
Credentials
Injects user/pass into the container without exposing in code
ECR
Docker images
Stores custom dump/restore images

Design Decisions

  • Fargate Spot for dump: Dump tolerates interruption (Step Functions retry handles it). ~70% savings.
  • Fargate On-Demand for restore: Restore cannot be interrupted mid-way (would corrupt the database).
  • Marker file: A JSON file that indicates "dump complete". Without it, EventBridge could trigger the restore before all databases are copied.
  • S3 Prefixes: Dumps are organized in  dumps/documentdb/  with naming convention  {database}_{timestamp}.tar.gz .



4. Implementation

📦 Full code (Docker, scripts, Terraform, Lambda) available in the repository:  documentdb-cross-account-dump-restore 
Below are the most relevant snippets for understanding the solution.

4.1 Docker Image — Dump

The image is based on  debian:bookworm-slim  with  mongodump ,  mongosh  and AWS CLI installed:
FROM debian:bookworm-slim
RUN apt-get update && \
apt-get install -y wget gnupg python3 python3-pip ca-certificates curl && \
rm -rf /var/lib/apt/lists/*
RUN pip3 install --no-cache-dir --break-system-packages awscli
RUN wget -qO - https://www.mongodb.org/static/pgp/server-7.0.asc | apt-key add - && \
echo "deb http://repo.mongodb.org/apt/debian bookworm/mongodb-org/7.0 main" | \
tee /etc/apt/sources.list.d/mongodb-org-7.0.list && \
apt-get update && \
apt-get install -y mongodb-org-tools mongodb-mongosh && \
rm -rf /var/lib/apt/lists/*
COPY dump-docdb.sh /usr/local/bin/dump-docdb.sh
RUN chmod +x /usr/local/bin/dump-docdb.sh
CMD ["/usr/local/bin/dump-docdb.sh"]

4.2 Core Dump Logic (simplified)

The script lists databases, runs  mongodump  on each, compresses, uploads to S3, and creates a marker file that triggers the restore:
# Download TLS certificate (mandatory for DocumentDB)
wget -q https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem -O /tmp/global-bundle.pem
TLS_OPTIONS="--ssl --sslCAFile /tmp/global-bundle.pem --sslAllowInvalidHostnames"
# List databases (excluding admin and local)
DATABASES=$(mongosh "mongodb://$DOCDB_USER:$DOCDB_PASS@$DOCDB_HOST:$DOCDB_PORT/?retryWrites=false" \
$TLS_OPTIONS --quiet \
--eval "db.adminCommand('listDatabases').databases.map(d => d.name).join('\n')" \
| grep -v "^admin$" | grep -v "^local$")
# Dump → compress → S3
for DB in $DATABASES; do
mongodump --host="$DOCDB_HOST:$DOCDB_PORT" --username="$DOCDB_USER" \
--password="$DOCDB_PASS" --db="$DB" --out="/tmp/${DB}_${TIMESTAMP}" $TLS_OPTIONS
tar -czf "/tmp/${DB}_${TIMESTAMP}.tar.gz" -C /tmp "${DB}_${TIMESTAMP}"
aws s3 cp "/tmp/${DB}_${TIMESTAMP}.tar.gz" "s3://$S3_BUCKET/${S3_PREFIX}${DB}_${TIMESTAMP}.tar.gz"
done
# Marker file — signals dump is complete
echo "{\"timestamp\":\"$TIMESTAMP\",\"databases\":$COUNT,\"type\":\"documentdb\"}" \
| aws s3 cp - "s3://$S3_BUCKET/${S3_PREFIX}dump_complete_${TIMESTAMP}.marker"

4.3 Core Restore Logic (simplified)

The restore drops the database on the destination and runs  mongorestore :
# Drop and restore each database
mongosh "mongodb://..." $TLS_OPTIONS --quiet --eval "db.getSiblingDB('$DB_NAME').dropDatabase()"
mongorestore --host="$DOCDB_HOST:$DOCDB_PORT" --username="$DOCDB_USER" \
--password="$DOCDB_PASS" --db="$DB_NAME" $TLS_OPTIONS "/tmp/restore/${DB_NAME}/"

4.4 Step Functions — Orchestration

Step Functions uses  ecs:runTask.sync  to execute the Fargate task and wait for completion, with automatic retry on failure:
resource "aws_sfn_state_machine" "dump_docdb" {
name = "myapp-docdb-dump-ambiente"
definition = jsonencode({
StartAt = "DumpDocumentDB"
States = {
DumpDocumentDB = {
Type = "Task"
Resource = "arn:aws:states:::ecs:runTask.sync"
Parameters = {
Cluster = aws_ecs_cluster.dump.arn
TaskDefinition = aws_ecs_task_definition.dump_docdb.arn
CapacityProviderStrategy = [{ CapacityProvider = "FARGATE_SPOT", Weight = 1 }]
NetworkConfiguration = { AwsvpcConfiguration = { Subnets = var.private_subnet_ids, SecurityGroups = [aws_security_group.ecs_dump.id], AssignPublicIp = "DISABLED" } }
}
TimeoutSeconds = 7200
Retry = [{ ErrorEquals = ["States.TaskFailed"], MaxAttempts = 2 }]
Next = "DumpSuccess"
}
DumpSuccess = { Type = "Succeed" }
}
})
}

4.5 EventBridge — Automatic Trigger

EventBridge detects the marker file in the destination bucket and triggers the restore Step Function:
resource "aws_cloudwatch_event_rule" "s3_marker_docdb" {
name = "myapp-docdb-marker-created-ambiente"
event_pattern = jsonencode({
source = ["aws.s3"]
detail-type = ["Object Created"]
detail = {
bucket = { name = [aws_s3_bucket.dump_destino.bucket] }
object = { key = [{ wildcard = "dumps/documentdb/*.marker" }] }
}
})
}
🔗 Full code in the repository:  github.com/felipethebrito/documentdb-cross-account-dump-restore  — Dump/restore scripts, Dockerfiles, Terraform (IAM, ECS, Step Functions, EventBridge, Lambda, S3, Secrets Manager) — ready to adapt to your environment.

4.6 IAM Roles

The solution requires several IAM roles following least-privilege principle. Here's a summary of each role and its permissions:
Role
Service
Permissions
Purpose
ECS Task Execution
ECS
 AmazonECSTaskExecutionRolePolicy  +  secretsmanager:GetSecretValue 
Pull Docker image from ECR and inject secrets into the container
ECS Task
ECS
 s3:PutObject ,  s3:GetObject ,  s3:ListBucket  +  secretsmanager:GetSecretValue 
Upload/download dumps to/from S3, read credentials at runtime
Step Functions
States
 ecs:RunTask ,  ecs:StopTask ,  ecs:DescribeTasks ,  iam:PassRole ,  lambda:InvokeFunction 
Orchestrate ECS tasks and invoke validation Lambda
S3 Replication
S3
 s3:GetReplicationConfiguration ,  s3:ReplicateObject ,  s3:ReplicateDelete 
Cross-account/cross-region replication from source to destination bucket
EventBridge
Events
 states:StartExecution 
Trigger the restore Step Function when marker file arrives
Lambda Validator
Lambda
 s3:GetObject ,  s3:ListBucket  +  AWSLambdaBasicExecutionRole 
Read marker file and list dump files to validate replication completeness
Key point: The S3 Replication role in the source account needs permissions on the destination bucket (cross-account). The destination bucket policy must explicitly allow the source account's replication role to write objects.
Full Terraform code for all IAM roles is available in  origem/iam-docdb.tf  and  destino/iam-docdb.tf  in the repository.



5. Orchestration and Scheduling

Manual Execution (on-demand)

aws stepfunctions start-execution \
--state-machine-arn arn:aws:states:sa-east-1:111111111111:stateMachine:myapp-docdb-dump-ambiente \
--region sa-east-1 --profile ambiente

Automatic Execution (scheduled)

To schedule via EventBridge (e.g., every Sunday at 3am):
resource "aws_cloudwatch_event_rule" "weekly_dump" {
name = "myapp-docdb-dump-weekly"
schedule_expression = "cron(0 6 ? * SUN *)" # 3am BRT = 6am UTC
}

Monitoring

# Real-time dump logs
aws logs tail /ecs/myapp-docdb-dump-ambiente --follow --region sa-east-1
# Restore logs
aws logs tail /ecs/myapp-docdb-restore-ambiente --follow --region us-east-2
Step Functions also provides visual monitoring in the AWS Console — each step shows success/failure with timestamps.



6. Results

Metric
Before (manual)
After (automation)
Engineer time
~2 hours
0 (1 command or automatic)
Total time (dump→restore)
~2-3 hours
~1-2 hours
Success rate
~90% (human errors)
100% (with automatic retry)
Cost per execution
~$50 (engineer's hour)
~$0.10 (Fargate + S3)
Visibility
None
CloudWatch + Step Functions
Databases supported
Manual per DB
All automatically



7. Lessons Learned / Gotchas

What went wrong the first time

  1.  retryWrites=false  — The connection failed silently without this flag. It took 1 hour to discover that DocumentDB doesn't support retry writes.
  1. Unsupported indexes —  mongorestore  returned exit code != 0 even when documents were successfully restored (only indexes failed). We needed to treat this as a warning.
  1.  dropDatabase()  timeout —  dropDatabase()  on large databases took over 30s. We added  timeout 30  to prevent the script from hanging.
  1. Expired TLS certificate — We used a fixed certificate in the Docker image. We switched to downloading  global-bundle.pem  at runtime (always up to date).
  1. S3 Eventual Consistency — The marker file arrived before the  .tar.gz  files in the destination bucket (replication). The validation Lambda needed retry with sleep to handle this.
  1. 100GB ephemeral storage — Large databases need disk space in the container. Fargate's default is 20GB — we needed to explicitly set  ephemeral_storage { size_in_gib = 100 } .

DocumentDB Gotchas

  • It's not MongoDB — Tools that work with MongoDB may not work with DocumentDB
  • No  mongos  — No native sharding
  • Incomplete  changeStreams  — Limitations in CDC
  • Wire protocol version — Some MongoDB 5+ features don't exist
  • Direct connection required — No SRV records ( mongodb+srv://  doesn't work)



8. Conclusion and Next Steps

Value delivered

  • Zero human intervention — Dump + replication + validation + restore happen automatically
  • Cross-account/cross-region — Source in São Paulo, destination in Ohio, separate accounts
  • Infrastructure as Code — Everything in Terraform, reproducible and versioned
  • Negligible cost — ~$0.10 per execution vs ~$50 of engineer time
  • Reusable — The same pattern can be adapted for other databases (RDS, Aurora) with minimal changes

Planned improvements

  • [ ] Automatic weekly scheduling via EventBridge cron
  • [ ] Selective restore (choose specific databases)
  • [ ] Sensitive data masking in the destination environment (see section 9)
  • [ ] Slack/SNS notification when dump/restore completes
  • [ ] Parallelism in restore (multiple simultaneous ECS tasks)

Shared resources

The infrastructure was designed to share resources where possible (ECS clusters, S3 buckets, VPC), while maintaining independence in DocumentDB-specific components (task definitions, step functions, lambdas). This reduces cost and complexity.



9. Data Masking in the Destination Environment

A natural evolution of this automation is ensuring that sensitive data from the source environment (PII, emails, national IDs, phone numbers) is not exposed in the destination environment. We added a post-restore step that executes masking queries directly on the destination DocumentDB.

Approach

After  mongorestore  completes successfully, an additional script connects to the destination DocumentDB and executes updates on collections with sensitive data:
// Sensitive data masking - post-restore
db = db.getSiblingDB('my_database');
// Mask emails and phone numbers
db.users.updateMany({}, [{
$set: {
email: { $concat: ["user_", { $toString: "$_id" }, "@masked.local"] },
phone: "11999999999",
document: "00000000000"
}
}]);
// Mask financial data
db.transactions.updateMany({}, [{
$set: {
account_number: "000000000",
card_number: "****-****-****-0000"
}
}]);
// Drop audit collections (not needed in destination)
db.audit_logs.drop();
db.access_logs.drop();

Integration in the Flow

Masking is added as a final step in the restore script. After each database is successfully restored, the masking script executes via  mongosh :
# After mongorestore
echo "Masking sensitive data..."
mongosh "mongodb://$DOCDB_USER:$DOCDB_PASS@$DOCDB_HOST:$DOCDB_PORT/$DB_NAME?retryWrites=false" \
$TLS_OPTIONS --quiet --file /usr/local/bin/mask-data.js
Or as a separate step in Step Functions:
ValidateReplication → WaitBeforeRestore → RestoreDocumentDB → MaskSensitiveData → Success

Considerations

  • Performance: Bulk updates on large collections can be slow. Use  bulkWrite  or process in batches for collections with millions of documents.
  • Idempotency: If masking runs twice, it should not cause issues — updates overwrite with the same masked values.
  • Validation: After masking, run a check query to ensure no real data remains (e.g., search for real email/ID patterns).
  • Selectivity: Not all collections need masking — only those containing PII. Maintain a list of collections and fields to mask.

Repository

The complete structure includes:
├── origem/ # Terraform — source account
│ ├── ecs-docdb.tf
│ ├── stepfunctions-docdb.tf
│ ├── iam-docdb.tf
│ └── secrets-docdb.tf
├── destino/ # Terraform — destination account
│ ├── ecs-docdb.tf
│ ├── stepfunctions-docdb.tf
│ ├── iam-docdb.tf
│ ├── eventbridge-docdb.tf
│ ├── lambda-validator-docdb.tf
│ └── lambda/index_docdb.py
└── docker/
├── dump-docdb/
│ ├── Dockerfile
│ └── dump-docdb.sh
└── restore-docdb/
├── Dockerfile
├── restore-all-docdb.sh
└── restore-docdb.sh


If you've implemented something similar or have suggestions for improvement, I'd love to exchange ideas in the comments!
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article