
Rank Without The Scan: How I Designed The Infinite Leaderboard
A deep technical walkthrough of my approach to AWS Builder Center's DynamoDB Football Data Modeling Challenge #4: what the brief actually required, the design decisions I made against each requirement and why, the alternatives I rejected, how I proved the throughput number for real, and a production bug I found and fixed along the way.
Rank Without The Scan: How I Designed The Infinite Leaderboard
AWS Builder Center's DynamoDB Football Data Modeling Challenge #4 asked for something specific:
a leaderboard platform on DynamoDB alone, supporting an unbounded number of named leaderboards,
each able to take over 1,000 score writes per second at peak, serving four access patterns
cleanly, no Redis, no MemoryDB. This is the full account of how I approached that, start to
finish: what the brief actually required, the design decisions I made against each requirement
and why, the alternatives I considered and rejected, how I built and tested it for real, and a
production bug I found and fixed along the way.
Challenge brief: https://builder.aws.com/content/3HvOSog8WSthnjgpXpi7yo1ghW2/the-infinite-leaderboard-creative-competition-at-global-scale

What the brief actually required
Before any design work started, I read the brief closely enough to write down exactly what it
asked for, separate from what a leaderboard "should" look like based on habit. The scenario:
fans create and join an open-ended set of leaderboards, "Best Prediction Streak," "Most Accurate
Lineup Builder," "Fastest Trivia Responder," anything the community dreams up. Each leaderboard
tracks scores per participant and must support more than 1,000 score updates per second at peak.
The three required deliverables, as stated:
- A DynamoDB table design (keys, indexes, item structure) that supports creating arbitrary leaderboards by name, recording scores per participant per leaderboard, and querying top-N rankings efficiently.
- An explanation of the partition key strategy for distributing writes across a high-velocity leaderboard without creating hot partitions.
- A strategy for serving four core access patterns: top N scores on a leaderboard, a specific participant's rank and score, neighboring records, and all leaderboards a participant is active on with their scores.
Three stretch goals, explicitly optional: showing users their current rank, supporting data types
beyond integer scores, and a cost analysis showing how on-demand capacity handles spiky traffic
without over-provisioning. I treated all three as required for my own bar, not just the core four,
since a design that only answers the mandatory list and ignores what the brief says it would love
to see is a weaker answer to "design rationale" than one that engages with the whole brief.
Every design decision below is organized around answering one of those requirements directly, with
the reasoning for why I chose it over the alternatives, because the brief's own judging weighs
data model correctness and design rationale as heavily as whether the thing runs.
Reviewing the design before writing infrastructure
That last point shaped how I worked from day one. I checked every load-bearing claim against
official AWS documentation instead of assuming it or from how another database would
solve the same problem, and the design went through five separate adversarial review passes before
a single line of CDK was written. Two of those passes found real bugs in the design on paper: a
bucket-counter key shape that looked right but wasn't actually range-queryable the way the access
pattern needed, and a transaction shape that would have issued two operations against the same
item, something DynamoDB rejects outright. Finding those before they became runtime bugs was the
entire point of reviewing the design before building it, not after.
Deliverable 1 and 2: write-sharded, not partition-per-leaderboard, and why deterministic
A single DynamoDB partition tops out around 1,000 WCU per second. One popular leaderboard can hit
that ceiling on its own, so the write path cannot live on one partition key, which is exactly what
the brief's second deliverable asks me to address directly.
The naive fix for a hot partition is a random suffix on the key. I rejected that specifically
because a leaderboard score is not a write-only event, it is a value you also need to read back
and update atomically:
ADD score :delta . If a participant's score item lives at a random shard
on every write, there is no single place to point a strongly consistent GetItem or a conditional
update at, and the update stops being atomic. So the shard has to be a deterministic function of
identity, not random: hash(participantId) % shardCount . A given participant's score always lands
on the same shard, so reads and conditional writes against that participant stay O(1) single-item
operations, while the aggregate write volume across all participants still spreads evenly across
shardCount physical partitions.Reading the top of a leaderboard becomes a scatter-gather across those shards: query every shard
in parallel, merge-sort in application code, take the top N. The important refinement here, and
the reason this stays cheap regardless of leaderboard size, is that each shard query only needs to
request its own top N, never its full contents, because no item ranked below position N on its own
shard could ever appear in the global top N. That bounds every top-N read to
shard_count × N
items fetched, completely independent of how many total participants exist on the leaderboard.Deliverable 3, part one and two: top N and a participant's own rank and score
Top N is the scatter-gather above. A participant's own score is a single
GetItem on the base
table, keyed by participant first ( PARTICIPANT#<id> ), so it costs the same whether the
leaderboard has ten participants or ten million.Rank is the sharp edge of this brief, and it is worth being precise about why. "What place am I
in" is not a Top-N query, it is "how many people are ranked ahead of me." Answered naively, that
means counting across every write-shard on every request, which means the cost of a rank lookup
scales with how many participants are on the leaderboard, exactly the thing sharding was supposed
to prevent in the first place. DynamoDB also has no ordinal operator: there is no "give me the
count of items ranked above this key" primitive, only key-range queries.
My answer, and the piece of this design I spent the most extra time on: don't count people, count
buckets. Every leaderboard gets a fixed set of score-range buckets (0 to 99, 100 to 199, and so
on), and each bucket keeps a running counter that updates in the same transaction as the
participant's score change. Finding a rank becomes "sum every bucket counter above mine," a fixed
number of reads bounded by bucket count, not by participant count.
PK = LB#<name>#BUCKETSHARD#<subshard> SK = BUCKET#<bucketId> count (ADD +/-1)The honest trade-off is that rank becomes approximate to the width of a bucket, not exact. I made
that call deliberately rather than defaulting to it. Anyone inside the already-computed top-N
slice still gets an exact rank for free, since that is just their position in the merged list.
Everyone else gets "somewhere in this band," which is the right answer for a leaderboard UI and a
flat-cost answer for the database. A rank lookup now costs
O(bucket-shards) , forever, not
O(shard-count) per request the way a naive per-shard count would, and not O(participants) at
all.That is also the shape that ships in the actual UI: top 10, your exact standing, and your nearest
neighbors, all from bounded queries, all live.

Deliverable 3, part three and four: neighbors and cross-leaderboard profile
Neighboring records reuse the same scatter-gather as top N, but with a numeric range condition
centered on the participant's own score instead of a fixed "top of the list" starting point,
widening the window and re-querying if either side comes back short. This is where the native
composite sort key (below) matters most: the range condition runs directly against the real
score attribute, no sentinel construction or string-boundary math required.The fourth pattern, all leaderboards a participant is active on, is the reason the base table is
keyed by participant first rather than by leaderboard first.
PARTICIPANT#<id> as the partition
key means "every leaderboard this person is on" is a single Query against one partition, no GSI
needed, no cross-leaderboard fan-out. This was a genuine either-or in the base table design: keying
by leaderboard first would have made the write-sharding story slightly simpler, but it would have
made the cross-leaderboard profile pattern expensive. I chose participant-first specifically
because the brief lists all four access patterns as equally required, not just the first two.A native composite key instead of a hand-rolled one
One design decision I changed late: DynamoDB shipped
multi-attribute composite sort keys in November 2025, meaning a GSI sort key can natively be
(score, participantId) as two real attributes, sorted and range-queried correctly, including
negative scores, with no encoding step required. Earlier drafts of the design hand-padded scores
into fixed-width strings to get correct sort order, the classic workaround for this exact problem.
Switching to the native feature removed the encode and decode functions entirely, along with a
class of bug that comes with them: get the padding width wrong once, or forget to invert it for
descending order, and negative scores or "lower wins" leaderboards silently sort backwards.Stretch goal: supporting more than integer scores
The brief's second stretch goal asks for data types beyond a plain integer. I generalized score to
a sortable rank value rather than hardcoding integer semantics into the query logic. A
sortDirection flag on the leaderboard's metadata item ( ASC or DESC ) drives which way the
ranking queries read the index, so "lower wins" leaderboards like race times work exactly the same
way as "higher wins" point totals, no special-casing in the read path. The composite sort key also
supports up to four attributes total, so a tiebreaker is simply a third native attribute
( score , tiebreak , participantId ), still natively and correctly sortable, still no string
concatenation. Non-numeric primary values, a formatted race time for example, are stored as their
sortable numeric equivalent (milliseconds) for ranking, with the display-formatted string kept as
a separate, non-indexed attribute. Ranking always happens on a comparable number; formatting is a
read-time concern, not a storage concern.Stretch goal: the cost story, and why on-demand
The brief's third stretch goal asks specifically for a cost analysis of on-demand capacity against
spiky traffic: quiet between matches, explosive during live play. I took this as seriously as the
required deliverables, because it directly tests whether the design rationale holds up under a
real constraint, not just a data-modeling one.
AWS's own capacity-mode guidance maps this exact traffic shape, a step-function spike followed by
sustained high load, to a recommendation of on-demand, not provisioned-with-autoscaling. The reason
is concrete, not just a rule of thumb: AWS's own documentation describes a two to five minute
reaction latency between throttling onset and new provisioned capacity actually coming online,
meaning a real step-function spike from a live match can produce a multi-minute outage window
under provisioned autoscaling before it catches up. On-demand sidesteps that specific failure mode.
I did not treat on-demand as a free pass, either. Its real limit is that it auto-scales to at most
twice the previous recorded peak within a 30 minute window, so a genuinely viral moment that jumps
past 2x needs pre-provisioned warm throughput ahead of a scheduled marquee match, not "on-demand
and hope." That is called out explicitly in the design rather than glossed over.
The other thing I worked through rather than left implicit: what a single score change actually
costs.
TransactWriteItems bills at twice the normal WCU cost of every item it touches, and a
score-changing update already costs more than one WCU once the GSI's key-change amplification is
counted (a delete plus an insert on the index). Combined, a single score change costs:Case | Items in the transaction | WCU after the 2x multiplier |
|---|---|---|
Same-bucket update, the common case | 1 (score item only) | 6 WCU |
First write | 2 (score item plus one bucket counter) | 8 WCU |
Bucket-crossing update | 3 (score item plus two bucket counters) | 10 WCU |
At the brief's stated floor of 1,000 writes per second, aggregate demand across the system is
therefore roughly 6,000 to 10,000 WCU per second depending on how often score changes cross a
bucket boundary, materially higher than the "1,000 WCU per second" a naive reading of the brief
would suggest. That is the number that should actually size warm-throughput and cost estimates,
not the raw write-count floor, and it is the kind of gap that only shows up when you work the
transaction cost through explicitly instead of assuming one write equals one WCU.
Alternatives I considered and rejected
Design rationale is one of the brief's own judging dimensions, so I treated naming what I did not
build, and why, as part of the deliverable rather than an afterthought.
- Redis or MemoryDB. AWS's own gametech guidance is honest that Redis sorted sets are the right tool for true O(log N) exact rank. I stayed DynamoDB-only for two reasons: the challenge is explicitly a DynamoDB data-modeling exercise, so reaching for a second stateful service would not actually engage with the brief, and the bucket-counter design already gets bounded-cost approximate rank without one. If exact real-time rank at Redis-grade latency ever became a hard product requirement, the honest next step would be a hybrid: DynamoDB as the source of truth, MemoryDB sorted sets as a Streams-fed derived serving layer for just the rank and neighbors hot path. I named that as the next architectural step this design would recommend if the constraint were lifted, rather than quietly building around the brief's own stated constraint.
- A periodic materialized snapshot. Streams feeding a tumbling-window job that recomputes and writes a full sorted leaderboard every few seconds is simpler than the bucket-counter approach and gives exact rank as of the last snapshot. I rejected it because "exact but stale by up to one window" is a worse fit for a leaderboard than "always approximate but continuously current." A snapshot answer is wrong precisely in the seconds right after a score changes, which is exactly when a competitive participant is most likely to check their rank. Bucket counters update inside the same transaction as the score itself, so there is no staleness window, only a bounded precision error.****
- A real-time OLAP sidecar, such as Rockset paired via Streams. This is a pattern AWS itself has documented for exactly this class of problem, on the reasoning that key-value stores were not designed for arbitrary aggregation queries. It is arguably a closer fit than the Redis comparison above, since it is purpose-built for rank-style aggregation rather than a general sorted-set primitive. I did not adopt it for the same underlying reason as Redis: it is a second stateful service the brief does not ask for, and the bucket-counter design already answers the access pattern without one.
- DAX, DynamoDB's own caching layer, is a genuinely good fit for one specific hot path here: the top-N query is the single most repeated read in the entire system, since every participant viewing a leaderboard hits the same shared slice. I did not adopt it as part of the baseline because it is a pure performance layer on top of an already-correct data model, not something the data modeling itself requires, and it adds a running cluster with its own capacity planning and cost. It is named here as the natural first addition if top-N latency or read cost ever becomes the actual constraint in production, an informed deferral rather than an oversight.
Two real systems that shaped how I thought about this
Two production leaderboard systems, genuinely comparable in scale to what this brief describes,
changed how I reasoned about the design rather than just confirming it.
- Dream11, a fantasy sports platform handling over a million requests per second during live matches, migrated off manually-sharded relational infrastructure after a real outage during a 2020 IPL semifinal. Their own postmortem is explicit that the failure was not insufficient write sharding, it was shard-failure blast radius: one shard going down locked out a fixed slice of users during a live lineup-edit window. DynamoDB sidesteps that specific failure class by construction, since its partitions are fully managed and independently replicated, with no manually-owned shard-node topology whose failure takes a fixed slice of leaderboards down with it. Write-sharding in my design is purely a throughput decision, not an availability one, which is a meaningfully different risk profile than Dream11's original architecture had.
- Duolingo's Leagues is the closest product-level analog to "many concurrent leaderboards, high write volume, tolerance for approximate rank," and it takes a path I deliberately did not: it avoids the unbounded-leaderboard problem entirely by keeping each league a small, fixed-size pool rather than one arbitrarily large leaderboard. That is a legitimate answer when a product can bound leaderboard size. This brief explicitly asks for an unbounded number of named leaderboards with no stated per-leaderboard size cap, so that option was not available to me, and I named it as deliberately not taken rather than simply not considered.
Stretch goal: live rank-change push
Beyond the required patterns and the three named stretch goals, I built a fourth thing because a
design on paper and a system that actually runs are different claims: a DynamoDB Stream watches
for writes that cross a rank-bucket boundary, an EventBridge rule catches that event, and a Lambda
pushes it over a WebSocket API to every browser currently on that leaderboard. A rank change shows
up as a toast in under a second, not on the next poll. The WebSocket API is deliberately left
unauthenticated, unlike the write path: it only ever pushes already-public leaderboard data, so
there is nothing to protect there that the IAM-authenticated write path does not already protect
upstream.
Proving the throughput number, not asserting it
The brief's bar is concrete: over 1,000 writes per second, sustained, on one leaderboard. I did not
want to leave that as a claim on a diagram, so the live demo has a real "Scale test" panel that
fires N distinct, real participant identities at the loaded leaderboard concurrently and times only
the write phase.

A separate, methodologically stricter run, 20 genuinely independent Lambda invocations instead of
one browser tab, hit 999 out of 999 succeeded, cross-checked directly against CloudWatch: zero
throttled requests, zero system errors, with bursts up to 6,062 WCU consumed in a single one
minute window. Here is that write capacity spike against the deployed table, next to the console's
own "write throttled requests: no data available":

Getting to that number was not clean, and the dead ends are worth writing down honestly.
The first bottleneck was not DynamoDB, it was an AWS account quota. My account had a
10-concurrent-execution Lambda limit in one region versus 1,000 in another, purely because of
which region had prior usage history. Nothing to do with the data model, everything to do with
where the stack happened to be deployed. Moving the whole stack to the higher-quota region was the
fix. DynamoDB itself never throttled a single request at any point in this process, at any
concurrency level tested.
The second bottleneck was the client, not the server. SigV4-signing every request on a single
JavaScript thread serializes what should be concurrent writes. Fanning the signing work across
real Web Worker threads is what took the browser-based scale test panel from roughly 350 writes a
second to 930 writes a second in the densest single second, at 1,000 concurrent writers, with
every write succeeding.
A bug I found in production, not in review
The live-push stretch goal did not work the first time it was deployed. No errors on the write
path, no errors on the API. The push Lambda just silently never fired. I went to CloudWatch Logs
Insights and found the actual reason sitting there, timestamped:

The stream-processing Lambda had permission to read the DynamoDB stream and permission to publish
to EventBridge, but not plain
dynamodb:GetItem on the table itself, which it needs to look up a
leaderboard's bucket configuration before it can decide whether a write actually crossed a rank
boundary. One line fixed it: table.grantReadData(rankStreamFn) . The interesting part is not the
fix, it is that a design doc, a code review, and a cdk synth will all wave a gap like this
through, because every individual IAM grant looks correct in isolation. Only the running system,
and its logs, told the truth. I verified the fix by opening a raw WebSocket connection directly
from the browser console, independent of the app's own client code, and watching a real push
arrive.Fine-grained access control, since this is a real system
Every write in this system is scoped by IAM to the calling participant's own partition, using a
dynamodb:LeadingKeys condition tied to the caller's Cognito identity, so the enforcement is not
just application logic trusting whatever participantId a client happens to send. This was not
part of an earlier draft of the design and came out of treating the brief as describing a real
system rather than a modeling exercise on paper: a leaderboard platform where anyone could write
scores under someone else's identity would not actually be a working answer to the brief, whatever
its data model looked like.What I'd revisit first
Shard count and bucket width are fixed at leaderboard creation time in this design. A leaderboard
that gets radically more popular than it was provisioned for does not reshard itself. That is the
next thing I would tackle: probably a background job that watches write-throttling metrics per
leaderboard and re-shards past a threshold, rather than trying to make sharding fully dynamic on
every write, which would reintroduce the hot-partition problem this whole design exists to solve.
The short version
- I read the brief and its judging weight on design rationale first, and let five rounds of adversarial review catch design bugs before any infrastructure was written.
- Writes are sharded deterministically, not randomly, so a participant's own score stays a single atomic item even while the leaderboard's aggregate write volume spreads across many partitions.
- Rank is a bounded bucket count, not a scan or a per-shard count, so its cost stays flat regardless of leaderboard size.
- Scores use a native multi-attribute composite sort key instead of a hand-rolled string encoding, and generalize to any sortable value with an optional tiebreaker.
- The cost analysis works the real per-write WCU cost through explicitly, including transaction and GSI amplification, rather than reading "1,000 writes/sec" as "1,000 WCU/sec." Redis, a materialized snapshot, an OLAP sidecar, and DAX were all considered and named as deliberate non-choices, with the reasoning for each.
- A live rank-change push pipeline runs beyond the required and stretch goals, over Streams, EventBridge, and WebSocket.
- Every throughput number above is measured against the deployed system, not asserted on a diagram. The live demo's own Scale Test panel is reproducible by anyone reading this.
- A real production bug, a missing IAM read permission, was found through CloudWatch logs and fixed, and I am writing that down rather than leaving it out.
The full design writeup, the access pattern matrix, and the CDK source are all in the repo linked
above.
If you are working on this same challenge, I would like to hear how you approached rank
specifically. It is the part of this brief that looks simple until you actually try to make it
cheap at any scale.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article