Beyond Progressive Disclosure: Sharding Agent Skills
Progressive disclosure says when skill content loads, not how much. Sharding cut a typical agent read from 21,000 tokens to 1,300. Plus what sharding costs.
Series: GenAI (3 articles)
- 3Beyond Progressive Disclosure: Sharding Agent Skills This article
Beyond Progressive Disclosure: Sharding Agent Skills
A skill library for a coding agent hits a budget problem early. A few hundred procedures is more text than any context window should hold. The standard answer is progressive disclosure: load a summary, then load detail only when needed.
We had that, and it worked. The library still wasted tokens at a scale the usual metric does not show, because the usual metric measures the wrong thing.
The number that matters is not how much content loads. It is the cost of reaching one thing.
Three tiers, three different costs
Progressive disclosure in a skill library usually lands as three tiers.
- Tier 1, the index. One line per skill, always in the system prompt. The agent needs all of it to choose. You cannot avoid this.
- Tier 2, the entry point. The chosen skill's front page. Charged once per invocation.
- Tier 3, the body. Procedures, tables, schemas. Charged only when the agent opens the file.
Our library held roughly three-quarters of a million tokens of content, and Tier 3 was about 90% of it.
Count Tier 1 from what the runtime loads, not from what the configuration lists. A runtime may resolve skills from locations the configuration never names, and those skills are charged like any other. Ours came to about 42,000 tokens across roughly 900 skills. Configuration states intent; ask the running system what it loaded.
It is easy to call Tier 3 deferred and move on. "Deferred" describes when content loads. It says nothing about how much loads.
A file is as expensive as its worst read
Tier 3 is charged per read. So its real cost is the size of the smallest useful read.
One of our reference files held 21,000 tokens across 38 independent topics. To answer a question about one topic, the agent read all 38. It paid 21,000 tokens to use about 550.
That is a 38x overpayment. And it is invisible to any metric that reports total corpus size, because the file is still correctly deferred. It loads on demand. On demand, it loads everything.
We found 105 files costing over 2,000 tokens per read. Eighty-five of them held four or more independent topics. The overpayment ceiling across those 85 was about 370,000 tokens.
The worst offenders, by ratio:
- 21,000 tokens across 38 topics: 38x overpayment
- 15,000 tokens across 25 topics: 26x
- 10,700 tokens across 20 topics: 20x
- 13,800 tokens across 19 topics: 19x
- 12,300 tokens across 19 topics: 19x
How much does one read need?
Before sharding anything, we needed to know how agents read. A ceiling is not a saving.
A few skills in our library were already split into many small files. That gave us a natural experiment. We scanned 1,831 agent sessions and counted how many files a session opened from a given skill.
- mean files read per skill access: 1.98
- sessions reading exactly one file: 58%
- reading two or fewer: 78%
One skill was split into 18 files. A typical session touched 2.9 of them, about 16% of its reference content.
So a monolith charges roughly N/2 times what a read needs. The 38-topic file was charging about 19x on a typical access, not in theory but in practice.
Sharding the monoliths
The fix is sharding: split one large file into many small ones, each holding a single topic. We sharded 14 monoliths into topic-sized files.
Typical read cost, using the measured mean:
- 28,900 tokens becomes about 2,700, across 43 shards
- 21,800 becomes about 1,900, across 23 shards
- 21,200 becomes about 1,300, across 32 shards
- 12,300 becomes about 1,300, across 19 shards
- 9,800 becomes about 1,400, across 14 shards
Library-wide, read overpayment fell from about 370,000 tokens to about 243,000.
The rule that decides whether this works at all
Sharding a file does not help if the agent cannot tell which piece to open. An agent that cannot tell opens several files and pays more than the monolith cost.
So every shard set needs an index. One line per shard, naming what is in it, sitting at Tier 2 where the agent already reads.
This is the whole difference between our two examples. The already-sharded skill had an index, and sessions read 2.9 of its 18 files. The worst monolith had 28,900 tokens in two files, no index, and no directory of any kind. Every read was all or nothing.
An index also drifts. A shard set can gain files without gaining index entries, leaving them present on disk and absent from the only thing that could point at them. Nothing fails loudly. Put a completeness check in whatever runs on a schedule, because an index nobody verifies decays into the problem it was meant to solve.
We made the index mandatory in the tool, which cannot produce shards without one.
The index is not free. It sits at Tier 2, which charges per invocation, a few hundred tokens per skill. Across 14 skills it added about 4,600 tokens and bought a 127,000-token reduction in read overpayment. Every skill broke even inside a single read.
How to do it
The mechanics are simple enough to describe in full.
- Baseline all three tiers separately. One number for the whole library hides the problem. The tier that dominates is also the one charged least often.
- For each L3 file, count independently readable sections. Headings are a good proxy for topic boundaries. Divide the file size by the mean section size. That ratio is what a read overpays.
- Cut at heading boundaries, one topic per file. Never cut mid-section. Group small adjacent sections so no file is too small to be worth a tool call.
- Recurse when a file has few top-level headings. Some files carry three top-level sections and put their real structure two levels down. Splitting only at the top level leaves pieces as large as the original.
- Write the index in the same pass. One line per file, naming its topic, at the tier the agent already reads.
- Check the result before trusting it. Every anchor resolves and every relative pointer still has its target. Every file has balanced code fences and appears in the index.
Step 6 is the one worth building first.
What sharding costs
Sharding is not free, and its costs are the easiest part to leave unmeasured. Four of them are real.
Broken cross-references. A large reference file navigates itself. It says "see above". It uses anchor links to its own headings. Shard the file and those targets move to a sibling the agent has not opened. This is a correctness fault, not a cost one, and no token metric shows it.
Build the check for this before you shard, not after. A link checker that resolves every anchor and flags every relative pointer takes an hour to write. It is the only thing standing between a clean token saving and a quietly broken file.
One more round trip. Reading two files costs two tool calls where the monolith cost one. Tokens fall; wall-clock time might not. File I/O is not the cost: the difference between opening one file and two is under a millisecond. The cost is the extra model round trip.
That is measurable, and you should measure it rather than assume it. Per-turn telemetry gives you elapsed time, time to first content, tool-call count, and bytes of tool output. Elapsed time divided by tool-call count gives a crude per-call figure. Comparing turns that made one call against turns that made several isolates the marginal cost. In our environment a tool call sits in the tens of seconds, dominated by inference rather than transport. The marginal cost of one extra read is real but much smaller than that average.
The direction is genuinely unclear before you measure. An extra round trip adds latency. Reading 26,000 fewer tokens removes prefill work. Which wins depends on your model and your read patterns.
Ambiguous index entries. If two index lines describe themselves in similar words, the agent cannot choose between them and opens both. A checker can flag entries whose wording overlaps. Our largest shard set had six such pairs, and each one risks a wasted read.
Sharding too far. A 40-token shard spends a whole tool call to deliver almost nothing. That trades one waste for another. Group small adjacent sections to set a floor, and check the result: even with grouping, five of ours landed under 150 tokens.
When a file resists, cut on a different axis
One file saved only 34%. The reason it resisted reshapes the rule above, so it is worth following in detail.
It held 7,000 tokens under a single heading, so a heading-based splitter found nothing to cut on. But the structure was there. The file was a ten-step procedure written as a numbered list, and the tool only recognised headings.
The first fix is obvious: teach the splitter to treat top-level list items as boundaries too. Ten steps, seven hundred tokens each, done.
That fix would have been wrong. Look at how the content is actually used:
- eight steps hold about 2,450 tokens between them and run in sequence, every time
- one step holds 2,750 tokens of parallel diagnostic collection, and nearly all of it runs
- one step holds 1,780 tokens of conditional remediation, roughly seven branches, of which you execute one
An agent working this procedure reads the sequence. Splitting it into ten files means ten reads to do the work of one, which is strictly worse. But the remediation step is a branch table, and there the agent needs one branch out of seven.
So the right cut is not per step. Keep the sequential spine in one file and shard only the branching step, one file per condition. A run would then read the spine plus one branch: about 2,700 tokens against 7,000. That is a projected 62% saving, where per-step sharding would have made things worse. That figure is arithmetic from the token distribution rather than a measurement, because we have not made the change yet.
That generalises into a sharper rule. The precondition is not many independent topics, which is the obvious guess. It is content the agent reads selectively. Reference material qualifies because a question touches one topic. A branch table qualifies because one condition applies. A sequence does not qualify at all, however long it is, because it is all needed.
Cut where the content branches, not where it sequences. Measure which you have before you cut.
Two failure modes to guard against
Sharding is a text transformation on documents nobody wrote expecting them to be cut. Two failure modes deserve guards.
Headings inside code fences. Reference files often hold report templates or example output inside fenced code blocks, and those templates contain their own Markdown headings. A section splitter that matches headings with a plain regex will treat them as real boundaries. It cuts them out of the enclosing fence and leaves unbalanced fences on both sides. Make heading detection fence-aware, and assert that every output file has an even number of fence markers.
Size checks do not catch structural damage. A character-count reconciliation between input and output passes while the fence damage above happens. Every character is still present, only in the wrong place. Byte conservation is not structure conservation. Assert on structure, not on size.
Anchor links need the same care. A large reference file navigates itself with links to its own headings, and those targets move when the file is cut. Rewrite them to name the target file, and check that each one resolves. Anchor generation is also easy to reimplement incorrectly. Strip the punctuation but keep the whitespace it leaves behind, and a heading containing an ampersand anchors with two hyphens, not one.
Make it continuous, not a project
Sharding by hand is a one-time cleanup, and a library that grows will need it again. The durable version is a measurement that runs on a schedule and a generator that produces the right shape from the start.
If a pipeline writes your skills, that pipeline is the right place for both. Ours runs a token-counting pass every six hours across the library. Extending it from total size to read cost took one function. Count the independently readable sections in each file, take the mean, and report the ratio between the file and one section. The same pass can flag a shard set whose index is incomplete, which is the failure mode that turns the optimization upside down.
A generation pipeline can also supply a signal that a static scan cannot. Ours records, per skill invocation, whether the agent opened the reference content or the procedural file sufficed. Over a few hundred invocations that becomes a per-skill granularity number. That is far better than the library-wide average used here. One skill showed its reference content needed in only 14% of invocations. That is a strong argument for keeping it in a separate file from the procedure. Note that this is a different question from whether the reference file itself should be split further. A file can be worth separating and still be a poor sharding candidate internally.
The strongest move is earlier still. Generate skills already sharded, with the index as a structural requirement rather than a later pass. Treat an oversized reference file as a lint failure at creation time. Sharding after the fact recovers tokens. Generating the right shape means never spending them.
What we would tell you
Measure the cost of one read, not the size of the corpus. A correctly deferred file can still charge 38x what a question needs.
Measure how agents read before you shard. The 1.98 figure decided the granularity. Without a measurement, granularity is a guess.
Never shard without an index. The optimization inverts without one. This is the single rule we would keep if we could keep only one.
Build the regression check before the change, not after. Cross-references break silently, and a token metric will never show it.
Check a validator's own logic before you trust its verdict. People believe a tool that reports with confidence, so a bug inside one is expensive.
Automate the measurement, and shape skills correctly at creation. A scheduled read-cost pass catches drift. A generator that emits indexed shards means the debt never accrues.
Cut where content branches, not where it sequences. A long sequence is all needed, so splitting it adds reads. A branch table is one path out of many, so splitting it pays.
Record what did not work. One suspected saving, duplicated content shared between skills, turned out to be 162 tokens across the whole library. Ruling a lever out is a result worth writing down.
The headline here is not a percentage. It is that "loads on demand" and "loads a reasonable amount" are different properties, and it is easy to build a metric that only sees the first.
Series: GenAI (3 articles)
- 3Beyond Progressive Disclosure: Sharding Agent Skills This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article