
How We Halved Our Agent's Token Bill Without Losing Accuracy
Same model, same task, same quality bar: 55% fewer tokens. What we changed in the harness, measured on a real agent workload, and which of the eight moves port to yours.
The views and opinions expressed in this article are my own and do not represent the official positions of Amazon Web Services (AWS).
Agent orchestration has evolved fast. It started with structured output and tool calling: one call, a schema in, JSON out. ReAct turned the single call into a loop, where the model calls tools, reads results, and decides what to do next. Graphs split that loop across specialized subagents. At each step more of the work moved out of the prompt and into the system around the model, and that surrounding system, the tools, the orchestration, the error handling, has grown into an engineering discipline of its own: harness engineering.
Why it deserves the attention comes down to cost. A general-purpose harness will get the work done. Give a capable model a filesystem, a query tool, and a loop, and it finishes most jobs you point it at. What that setup doesn't control is what the job costs. As agents move from demos to production, tokens per task is becoming the number teams actually watch, because it decides whether an agent workload is a rounding error or a budget line, and two harnesses that produce the same output can differ on it by multiples.
This article is a practical account of that difference on one real system: Data Wiki, the harness we use to turn a data catalog into documentation an agent can read. We measured where its tokens went, re-engineered the harness around the model, and the cost of a harvest fell by more than half. The model didn't change, and neither did the quality of what it produced.
Data Wiki, briefly
Data Wiki reads an AWS Glue catalog and writes documentation for it. A harvester agent samples the real data and writes a wiki: markdown docs explaining what each table and column means, how tables join, and what values a column actually takes. Any agent can then read that wiki over MCP, the same way a new hire reads a well-kept internal wiki. We open sourced it as sample-okf-llm-wiki .
A quick refresher on token economics
An agent runs as a sequence of turns. On each turn the model reads the conversation so far, reasons over it, and either calls a tool or writes its answer. The model is stateless between turns, so "reads the conversation so far" is literal: every turn re-sends the system prompt, every earlier tool call, and every tool result as input tokens, then adds a small amount of new output on top.
That asymmetry is the core of agent economics. Output grows linearly, one increment per turn. Input grows quadratically, because each turn re-pays for every turn before it: by turn forty, the run has re-sent turn one's content thirty-nine times. What a task costs is decided far more by how many turns it takes, and how much context each turn drags along, than by the length of anything the agent finally writes.

Prompt caching softens the input side. Providers cache a prompt's prefix, so a turn that only appends to a stable conversation pays roughly a tenth of the fresh price for everything already cached. The discount has one condition: if anything earlier in the context changes, the cached prefix no longer matches and the run is back to full price. A harness that keeps context stable keeps the discount.
One dataset, two harnesses
We tested this on one dataset, Formula 1 from the BIRD mini_dev benchmark, harvested twice: once with a standard agent loop, once with a harness engineered specifically to control cost. A harvest turns the dataset's 35 tables into a wiki of around 75 markdown docs. Same tables, same model, same authoring skill, same acceptance bar for a finished doc. The only thing that changed between the two runs was the harness wrapped around the model: which tools it could call, what was computed in advance versus discovered live, and what was enforced by code instead of asked for in a prompt.
The naive harness spent about 100 million tokens on the full harvest. The engineered harness spent about 45 million, for the same harvest. That's a 55 percent drop, so "halved" is the modest version of that number.

Where the naive harness came from
The naive harness wasn't a strawman. It's a standard agent loop, the same shape as a coding agent like Claude Code or Kiro: the model calls a tool, reads the result, decides what's next. It reads and writes files, lists directories, runs SQL, and leverages subagents to distribute the work, all guided by a single authoring skill, a written methodology doc. Guardrails live in the prompt: rules the model is supposed to remember, not rules the system enforces. Nothing here is wrong. It just wasn't built for cost at this scale.
In practice the loop runs like this. A main orchestrator dispatches subagents to author the individual docs. Once the wiki takes shape, it sends out a fleet of reviewer subagents to read what was written and look for problems, and fix authors to act on what the reviewers find. A full harvest quickly amounts to well over a hundred subagent invocations, and those roles, orchestrator, author, reviewer, fix author, are the cast for the rest of this piece.
What we found before we changed anything
To optimize the harness we first had to know where the tokens go. Every agent invocation is traced turn by turn and tool call by tool call, and a script reduces those traces to a bill per agent role for one full harvest. The answer was clear: most of the spend sat in probe queries, live checks against the real data, and in authors and reviewers re-reading things the run had already read, not in the final writing.
The same join, verified three times.
results.driver_id joins to drivers.driver_id . One fact, checked independently by three parts of the system. The author of results.md probed it early, on a small context. The orchestrator probed it again later, on a much larger context. A reviewer probed it a third time. The same fact was bought three times, and because each buyer carried a bigger context, each purchase cost more than the last.
Three slow leaks, none of them fatal alone:
- Coordination chatter. Every orchestrator↔reviewer message piled into one context that only grew.
- Status by re-counting. Checking progress meant re-listing files and re-counting docs, billed on the whole context every time.
- Rules re-read forever. About 3,000 tokens of prompt rules got re-sent, unchanged, on every turn.
Each of these gets a fix below. The fixes group under three levers: smaller context, fewer turns, cheaper failures. The first two shrink what each turn re-sends as input. The third shrinks how often a run pays twice for the same work.
Smaller context
The biggest single change was to stop serving the data source through an API and start serving it as files. The naive harness asked its catalog questions one call at a time,
describe_table after describe_table , each call a turn, each result re-sent with every turn after it. The engineered harness snapshots all of the source metadata once, at run start, into a read-only directory of plain files, and agents explore it with the tools they are already strongest with, the same read, grep, and glob a coding agent lives in. One grep across columns.tsv for driver_id turns up the nine tables sharing that key in a single turn, and a looser scan of the same file surfaces near-matches, a foreign key spelled slightly differently across tables. Through per-table API calls, every one of those candidates would have cost its own round trip.Normalizing to files paid a second time, in the skill. Once every source looks like the same directory layout, the methodology stops caring where the metadata came from: a Glue catalog and a Redshift schema land in the same files, so the skill's always-loaded prose shrank to the part that is true everywhere. What genuinely differs by source lives in short sub-files the agent opens only when its dataset calls for them, progressive disclosure instead of a wall of upfront instructions. That's the third slow leak, the rules re-sent on every turn, cut at the source.
The third change packaged the whole review pass into a single tool. In the naive harness, review was a conversation the orchestrator ran personally: it reasoned about how to group docs for review, wrote instructions for each reviewer subagent (output tokens, the expensive kind), dispatched them, and collected the findings back into its own growing context. Now a
run_review tool does the pass as a black box. It clusters the docs deterministically by semantic connection, which docs link to and reference each other, dispatches reviewers per cluster in parallel with prompts built by code, and routes findings straight to a fix author. The orchestrator spends one call and reads one summary, and the first slow leak, coordination chatter, is gone.
Fewer turns
Most of what a table's documentation needs is deterministic: the column profile, whether a candidate join actually holds, the grain of the table, what one row represents. In the naive harness, agents worked all of this out live, probe query by probe query. That's exactly the heavy lifting a model doesn't need to do.
A precompute pass now runs before any agent exists. Plain Python, reusing the same SQL cores as the live tools, profiles every table and probes every join candidate once, and caches the results keyed by a fingerprint of the source. Authors start from evidence sheets instead of a blank slate: profiling done, joins inferred, and each claim carrying a verdict. HOLDS means the join or grain checked out. REFUTED means it demonstrably doesn't. WEAK means the evidence was too thin to trust outright. The author probes live only what no sheet covers.


The second turn-saver applies the same idea to the harvest itself. In the naive harness, "how far along are we" was a question the orchestrator answered with its own reasoning: re-list the docs, re-count coverage, compare against the table list, all billed against its full context every time it checked. Now a no-argument stats tool returns the whole picture in one cheap call: docs written, tables covered, joins recorded, issues open.
Verification got the same treatment, and it was one of the biggest wins in the run. Instead of agents spending turns reading docs back to assess them, a lint tool scans the entire bundle in one call and returns a list of concrete errors with locations:
Check | What it validates |
|---|---|
Coverage | every snapshot table has its doc; stale docs flagged |
Required docs | guardrails and a dataset overview exist |
Frontmatter | required keys present and consistent bundle-wide |
Links | every link resolves; orphan docs flagged |
Joins | named columns exist and their types compare |
Computations | shape valid; enums grounded in profile evidence |
EXPLAIN | every runnable SQL fence validated; no data scanned |
Cheaper failures
Some rules are compliance, not judgment: a doc's frontmatter needs a grain field, a join reference has to name columns that actually exist. The engineered harness stopped asking the model to remember rules like that and enforces them in a middleware guard every write passes through. A bad write bounces with the reason attached and costs one short turn to correct; a good write lands. Nothing malformed enters the bundle, so nothing malformed has to be hunted down later at review prices. And because the guard owns compliance, the skill spends its words on methodology instead of policing.

Same exam, half the bill
None of this is worth much if the cheaper wiki is a worse wiki. So we gave both harvests the same exam: the same question bank, the same grader, checking SQL result-set equality plus behavior checks that catch an answer that's technically correct but shallow, run several times each. Questions came from ground truth, never from either wiki, so neither harness could get credit for teaching to the test.
Accuracy held. No regression between the two wikis. Richness held too: coverage by concept type came out the same either way. The 55 percent didn't come out of the wiki's quality. It came out of turns the wiki never needed and context nothing was reading.

Putting it together
Where does this apply beyond one harvester? The honest boundary is task ambiguity. When the work is open-ended, research with no fixed goal, a bug hunt with no known shape, the agent's exploration is the product, and engineering a harness around it is a genuinely harder exercise: you can't precompute answers you can't predict, and you can't lint output you can't specify. But when tasks and objectives are defined, and most production agent workloads are exactly that, the same job run over and over against a known quality bar, engineering the harness can take you a long way on cost per task.
For that kind of workload, three principles travel well:
- Serve context as files and keep the always-loaded instructions lean. That's smaller context.
- Precompute what's deterministic and serve answers from cheap tools instead of model turns. That's fewer turns.
- Enforce correctness in code, at the moment of the write, instead of asking the prompt to police it. That's cheaper failures.
They break down into eight moves that port to any agent system:
Lever | Move | What it means |
|---|---|---|
Smaller context | Normalize sources to files | Snapshot external systems into a directory once; agents explore it with read, grep, and glob, the tools they're strongest at |
Smaller context | Slim the skill, disclose progressively | Always-loaded instructions cover only what's true everywhere; specifics sit in sub-files read on demand |
Smaller context | Package multi-agent passes as tools | A whole fan-out pass runs behind one call: deterministic grouping, code-built prompts, results routed in code |
Fewer turns | Precompute deterministic facts | Whatever code can derive, profiles, relationships, checks, runs before any agent and is cached across runs |
Fewer turns | Serve verdicts, not raw data | Agents get conclusions they can act on, not raw data to re-derive |
Fewer turns | Answer status with a tool | Progress and coverage come from one cheap call, never from re-counting inside the agent's reasoning |
Fewer turns | Lint in one pass | One scan of the full output points at every error, replacing turn-by-turn verification |
Cheaper failures | Enforce in middleware | Bad writes bounce with a reason before they land; prompts teach judgment, not compliance |
The short version: the model didn't get cheaper and the task didn't get smaller. The harness around them did. Keep context normalized and lean, precompute what doesn't need a model, put deterministic work behind cheap tools, and enforce structurally what a prompt can only request. Same job, same model, same quality bar, half the bill.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article