DMS Pro-Tip #1: Turbocharge DocumentDB DMS full load throughput with one simple setting
When migrating a large MongoDB or DocumentDB collection with AWS DMS, the full load is bottlenecked by the fact that each collection is read by a single thread, so one giant collection dominates the total migration time. By adding a single Table Mappings rule that splits that collection into 6 partitions to load concurrently, the migration time for a 180 GB migration shrinks from 51 minutes down to 16.
Series: DMS Pro-Tips (1 article)
- 1DMS Pro-Tip #1: Turbocharge DocumentDB DMS full load throughput with one simple setting This article
I've been working on techniques to make data migrations faster, including a "no touch" approach that needs no analysis or parameter tuning yet still performs well. While benchmarking it against a properly tuned AWS Database Migration Service (DMS) task, I discovered just how much improvement comes from a single DMS Task setting.
The challenge
Migrate a complete copy of every collection from a source MongoDB/DocumentDB instance to a target one, as quickly as possible.
Over the past five-plus years I've helped AWS customers with data migrations of all sizes. Speed isn't always critical, but faster is almost always better. The reason: every insert, update, and delete that happens on the source during the migration has to be replayed on the target afterward. The longer the full load takes, the more catch-up work there is.
The experiment
I wrote new code to auto-tune the full load process and wanted to see how it compared to DMS "out of the box" (point-and-click), doing little more than entering
%.% as the table selector (migrate every collection in every database).My test environment was a DocumentDB v5 cluster with roughly 180 GB of data across 23 collections in 12 databases:
- One 120 GB collection (the giant)
- Three 10 GB collections
- The rest each under 1 GB
The result: the stock DMS task took 51 minutes, while my new script finished in 16 minutes.
Why the difference?
By default, DMS loads each collection with a single thread, up to a maximum of 8 collections in parallel. That concurrency limit is configurable from 1 to 49, under:
Settings → Advanced Settings → Full Load Tuning Settings → Maximum number of tables to load in parallel
Raising it above 8 might shave a little time off, but it misses the real bottleneck: the 120 GB collection. No matter how many other collections load in parallel, that one is still being migrated by a single thread performing a full collection scan.
The one-setting fix
The fix is to parallelize the load of that giant collection, and it's easy. In the task's Table Mappings section, click Add new table settings rule and set:
| Field | Value |
|---|---|
| Source name | database containing the giant collection |
| Source table name | the giant collection's name |
| Number of partitions | 6 |
This splits the large collection into 6 partitions that load concurrently instead of one thread reading it end to end.
Watch the concurrency budget
The "maximum number of tables to load in parallel" setting includes these partitions. If you set the large collection to 8 partitions, it consumes the entire concurrency budget and nothing else loads until all 8 partitions finish.
By using 6 partitions, you leave room for two other collections to load at the same time. You can push throughput higher by raising the maximum-tables-in-parallel setting, just remember that each additional thread is another query running against your source instance.
The results
| Technique | Time (minutes) |
|---|---|
| DMS, no partitions | 51 |
| DMS, 6 partitions on the giant collection | 16 |
Conclusion
Your DMS full load performance may be gated by a lack of concurrency on your largest collections. Adding that concurrency is as simple as a single task change.
Got a DMS pro-tip of your own? Share it in the comments, I'll add it to my list.
Be sure to join the NoSQL Launchpad in Builder Center where AWS NoSQL specialists hang out, share knowledge, and post cool stuff.
Series: DMS Pro-Tips (1 article)
- 1DMS Pro-Tip #1: Turbocharge DocumentDB DMS full load throughput with one simple setting This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article