AWS Builder Center

Builder Center

Building a Leaderboard That Doesn't Fall Over: Write-Sharding and Scatter-Gather in DynamoDB

Building a Leaderboard That Doesn't Fall Over: Write-Sharding and Scatter-Gather in DynamoDB

A deep dive into designing a DynamoDB leaderboard that survives 1,000+ writes per second while staying instantly rankable — using write-sharding to spread the load and scatter-gather reads to put it back together. Covers the deterministic shard-assignment trick, the score-encoding scheme, and the trade-offs that come with the design.

Ajaykumar K V

AI Engineer | AWS Community Builder
There's a moment in every DynamoDB design where you hit a wall that feels less like "I don't know how to do this" and more like "I don't think this is possible." For me, that wall was: how do you sustain thousands of writes per second on a single leaderboard, while also being able to instantly answer "who's in first place?"
Those two requirements sound like they should coexist peacefully. They don't. And figuring out exactly why they don't — and what to do about it — turned into the most interesting data modeling problem I've worked through in a while. This is the story of that problem, the dead ends, and the design that came out the other side.

Table of Contents

  • The setup: a leaderboard, but make it hostile
  • The wall: sorting doesn't travel
  • Deciding what to pay for
  • How the sharding actually works
  • Putting the sort key back together
  • Reading a leaderboard that's scattered across shards
  • The one query that doesn't need any of this
  • What this looks like end to end
  • The honest accounting
  • Results

The setup: a leaderboard, but make it hostile

The idea is simple on the surface. Fans create leaderboards for anything — longest prediction streak, fastest trivia buzzer, most accurate lineup picks. Each leaderboard tracks a participant and a score. You need to be able to:
  • See the top scorers on any leaderboard, right now.
  • Look up any one participant's exact rank and score.
  • See who's just above and just below a given participant — the "you're 3 spots from the podium" view that makes leaderboards feel alive.
  • Pull up everywhere a single participant is competing, across every leaderboard they've touched.
None of that sounds unreasonable. The part that makes it hostile is the scale: any single leaderboard has to be able to absorb more than 1,000 score updates every second. Not 1,000 total. 1,000 per second, sustained, for as long as a match is live and a crowd is reacting to it in real time.
That number matters because DynamoDB has a very specific, very real ceiling: a single partition tops out at roughly 1,000 write capacity units per second. It doesn't matter if you're on provisioned or on-demand billing — that ceiling is a property of how a physical partition works, not a billing mode. If every score update for one leaderboard lands on the same partition key, you will hit that wall, and you will hit it well before you get anywhere near "viral leaderboard during a live goal" traffic.
So: spread the writes across more partitions. Easy, right?

The wall: sorting doesn't travel

Here's where it stops being easy. DynamoDB can sort items for you — but only within a single partition key. A sort key gives you ordering inside one partition's item collection. It gives you nothing about ordering across partitions. There's no native "give me the top 10 across all partitions, sorted" operation.
So now you're stuck between two requirements that actively fight each other:
  • To handle high write throughput, you need many partition keys for one leaderboard's data.
  • To answer "who's in the top 10" cheaply, you need one sorted collection for that leaderboard's data.
You cannot have both on the same key at the same time. This isn't a skill issue or a "just use a smarter index" issue — it's a structural property of how DynamoDB physically stores and sorts data. Once I actually internalized that, the rest of the design fell into place, because the real question stopped being "how do I avoid this trade-off" and became "which side of this trade-off can I afford to pay, and how do I minimize the bill."

Deciding what to pay for

I decided to pay on the read side. Writes get spread across shards; reads have to work harder to put the picture back together. Here's why that direction, not the other one:
The write path — a fan submitting a prediction, buzzing in on a trivia question, whatever generates a score — is the one access pattern explicitly required to survive 1,000+ operations a second. It has zero room for extra latency or extra hops. It needs to be one direct write, every time, no exceptions.
The read paths — checking the leaderboard, looking up your rank — can tolerate doing a bit more work per request, because even a "slow" read in this context is still tens of milliseconds, and nobody's read-side request rate approaches the write-side firehose during a live event. Reads also tolerate slightly stale data far better than writes tolerate dropped or throttled requests — seeing a leaderboard that's a few hundred milliseconds behind reality is a non-event; a throttled score submission is a fan's moment getting lost.
So: shard the writes, and make the reads do the reassembly work.

How the sharding actually works

Every leaderboard gets its own shard count, decided once at creation time, based on how much write traffic it's expected to take. The formula is almost embarrassingly simple:
1
ShardCount = ceil(expected_peak_writes_per_second / 700)
700, not 1,000. That's deliberate headroom below DynamoDB's real ceiling — it absorbs the fact that a hash function is uniform in expectation, not perfectly even in every finite sample, plus the extra write cost every score submission incurs by also updating a couple of indexes.
A quiet leaderboard — say, a niche community board with a handful of updates a minute — gets one shard and never thinks about this again. A leaderboard that goes viral during a live match and needs to handle 5,000 writes a second gets eight shards, each comfortably sitting around 625 writes/sec, safely under both the 700 target and DynamoDB's actual hard limit.
The part that took a couple of iterations to get right was how to assign a participant to a shard. My first instinct was to just pick a random shard on every write — maximally even distribution, dead simple. But that has a nasty consequence: if you don't control which shard a write lands on, you also don't know which shard to look in later. Reading or updating a single participant's own score would mean checking every shard until you found them.
The fix was to stop randomizing and start calculating: shard = hash(participant_id) mod ShardCount. Same participant, same shard, every time — deterministically, without needing to look anything up first. That one change turned "find my score" from an expensive multi-shard search into a single direct read: compute the hash locally, go straight to the right partition. Write throughput stays just as evenly distributed (participant IDs are diverse enough that the hash spreads them out fine), but now the single-participant operations — which are common, not rare — get to be cheap.

Putting the sort key back together

Sharding solves the write problem but it also creates a smaller problem: DynamoDB sort keys compare as plain strings, byte by byte. If you naively write a score as "9" vs "10", string comparison says "9" > "10" — which is exactly backwards from what you want.
The fix is an encoding step before anything ever touches a sort key. Every score gets scaled to a fixed number of decimal digits, shifted so negative scores become non-negative, and padded to a constant width — so that comparing the resulting strings byte-by-byte gives you the same answer as comparing the original numbers. For leaderboards where a higher score wins, the padded value gets inverted too, so that DynamoDB's cheapest, most natural query direction — reading forward, ascending — naturally returns the best scores first. No reverse-scanning trick needed on the hot read path.
That one encoding function quietly does double duty: because it treats every score as an arbitrary-precision decimal rather than assuming "integer," it works exactly the same way whether the leaderboard is counting points (higher wins), tracking a race time in milliseconds (lower wins), or comparing prediction-accuracy percentages. Same code path, same comparisons, no special casing for the type of number being ranked.

Reading a leaderboard that's scattered across shards

This is the part where the trade-off actually gets paid. Every read that needs a leaderboard's ranked view — top scorers, someone's rank, their neighbors — has to fan out across every shard in parallel and stitch the results back together in application code. Scatter, then gather.
The genuinely satisfying part was realizing that "scatter-gather" doesn't have to mean "read everything from everywhere." For a top-N query specifically, you only need to ask each shard for its own top N — because a shard's Nth-best entry can never be beaten by anything that shard didn't return. So pulling the top 10 from 8 shards means examining 80 items total, merging them, and keeping the best 10 — regardless of whether the leaderboard has a thousand participants or a hundred million. The read cost scales with the number of shards and the size of N, never with how big the leaderboard has actually grown.
Rank lookups don't get that same shortcut, and I want to be upfront about that instead of glossing over it. Answering "what's my exact position" means asking every shard "how many entries here beat mine," and adding up the answers — a query per shard, every time, regardless of whether the participant is near the top or buried near the bottom. It's the most expensive read in the whole system, and it's the part of this design I'd revisit first if read volume on rank lookups ever became the bottleneck instead of write volume on submissions — probably by building a small, periodically-refreshed cache of rank boundaries so most lookups get an approximate answer instantly and only fall back to the expensive fan-out when precision actually matters.
Neighbors — "who's just above and below me" — get a similar over-fetch-and-merge treatment, pulling a small buffer of extra candidates from each shard so the true neighbor set almost certainly gets captured even though no single shard knows the full picture. I'll be honest that this one is a probabilistic guarantee, not a mathematical one — a truly adversarial score distribution could theoretically need a bigger buffer than the default. In practice, real leaderboards don't cluster scores that way, but "in practice" is doing some work in that sentence and it deserved to be said plainly rather than implied.

What this looks like end to end

The honest accounting

I think the most useful thing I can leave here isn't "this design works," it's a clear-eyed list of exactly what it costs and where it would need to evolve:
  • Write sharding solves the throughput problem completely, at the cost of every ranked read needing to fan out across shards. That trade is worth it, because writes are the access pattern under the hard 1,000+/sec constraint, and reads have far more slack to absorb extra work.
  • Rank lookups are the sharpest edge of that trade — their cost scales with how many shards a leaderboard has, not with how popular it is or where someone ranks. It's the first thing I'd optimize if this had to scale another order of magnitude.
  • Shard count is decided once, at creation, and doesn't auto-adjust. A leaderboard that wildly outgrows its initial forecast would need an explicit re-sharding step — a solvable, bounded migration, but a real operational task, not something that happens automatically today.
  • Eventually consistent reads, used deliberately, everywhere they're safe to use — leaderboard views can lag the true state by a fraction of a second, and that's a fair price for not doubling the read cost of the single most frequent operation in the system.
  • On-demand billing isn't just convenient here, it's the cheaper choice — this traffic shape is naturally quiet for hours and explosive for minutes, which is exactly the pattern pay-per-request pricing is built for, and exactly the pattern that makes pre-provisioned capacity either wasteful or dangerously tight.
None of that is a flaw hiding in the design. It's what happens when you actually name the trade-off instead of designing around it and hoping nobody asks.

Additional Resources

# amazon-dynamodb# aws-cloudformation# nosql# data# databases
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article