AWS Builder Center
Your AI Agent’s Token Bill Is Not the Same as Token Waste

Your AI Agent’s Token Bill Is Not the Same as Token Waste

A new measurement framework separates necessary agent work from removable reasoning, repeated fetches, padding, and detours, giving builders a better way to reduce cost without cutting capability.

An AI agent’s token bill rises for two very different reasons. The task may genuinely require more reasoning, tool calls, and evidence. Or the agent may be repeating context, fetching the same data, narrating obvious actions, and exploring paths that never affect the result.
A cost dashboard cannot distinguish those cases. It can show that one workflow consumed more tokens than another, but it cannot tell an engineering team which tokens were necessary. If the response is simply to shorten every prompt, lower every reasoning budget, or switch every task to a cheaper model, the team may remove useful work while preserving the system behavior that caused the waste.
A recent paper, How Do AI Agents Waste Your Money?, offers a better starting point: treat token waste as a removability question. Content is waste when it can be removed from an agent trajectory without eliminating useful work. That definition turns a vague cost complaint into something engineers can test.

Four patterns inside an agent trajectory

The paper separates four recurring patterns.
Restate repeats information already present in the context. An agent may reproduce a policy, paraphrase an unchanged tool result, or announce the same plan before several actions.
Duplicate-Fetch calls a tool again for information that is already available and has not changed. Reading the same file twice and repeating an identical database lookup are examples.
Padding generates text that does not advance the task. Preambles, sign-offs, repeated descriptions of the request, and narration immediately before a tool call fall into this category.
Detour describes exploration that looked reasonable at the time but did not contribute to the eventual result. A failed approach can be expensive without being obviously irrational when the agent chose it.
That last distinction matters. The researchers classify restatement, duplicate fetches, and padding as strict waste. They treat detours separately because exploration is sometimes part of solving an unfamiliar problem. A production team should therefore resist labeling every unsuccessful step as waste. Otherwise, it will train or constrain the agent to avoid useful investigation.

Figure 1: A practical audit separates measurement, classification, and intervention. Original visual, CC BY 4.0.

Measure removal, not verbosity

The proposed framework has three stages: trace, identify, and verify.
First, it captures complete trajectories and normalizes model responses, reasoning, tool calls, observations, and token accounting. Second, an LLM-based judge marks candidate waste spans and maps them back to the original trajectory for token-level measurement. Third, the researchers remove the marked content, resume execution, and compare task outcomes and subsequent actions.
The verification step is the important part. A shorter trace is not automatically a better trace. If removal materially shifts the next-action distribution or lowers task success relative to a control, treat the passage as potentially useful even if it looked repetitive.
Across the six models and four benchmarks evaluated in the paper, the authors report that roughly 18% of output tokens were identified as waste. For more than 45% of tasks, the wasted output from one agent exceeded the entire output of another agent that successfully solved the same task. In separate online experiments with Kimi-K3 on SWE-bench Verified and MCP-Atlas, the framework removed 41–46% of model reasoning while largely preserving task performance.
Those are source-reported research results, not universal production ratios. They show that removable work can be material and measurable. They do not establish that every enterprise agent can cut its bill by the same percentage.

Why a token total points to the wrong fix

The paper also found that waste varied more across tasks and environments than across models. Longer contexts did not consistently produce a higher waste ratio. In the paper’s controlled Kimi-K3 study across τ²-bench (airline and retail), SWE-bench Verified, Claw-Eval, and MCP-Atlas, the highest reasoning setting generated 6.6 times as many reasoning tokens per trajectory as the low setting, yet 66% of that reasoning was still judged non-wasteful on average.
This creates an architectural warning. Total tokens, context length, and reasoning effort are consumption measures. None of them identifies the cause of unnecessary work on its own.
A blanket context limit may delete evidence that later actions need. A lower reasoning setting may shift waste into verbose tool interactions. A cheaper model may repeat calls because it has more difficulty selecting tools. Prompt caching may lower the cost of repeated prefixes while leaving duplicate actions and dead-end execution untouched.
The optimization target should be successful task completion with less removable work, not the smallest possible transcript.

A production audit for agent waste

Start with a bounded task class and an outcome that can be checked. Examples include a resolved service request, a passing code change, or a completed data-reconciliation workflow. “Helpful answer” is too subjective for an intervention test.
Record the complete trajectory under a stable task and run identifier. Include each model invocation, tool name and arguments, observation, retry, token count, cache activity, latency, result, and termination reason. Apply the same privacy, access, and retention controls used for other sensitive application telemetry. Agent traces can contain user inputs and tool results.
Classify candidate waste by mechanism. Strict categories can often produce direct engineering actions:
  • Restatement points to state representation and prompt design.
  • Duplicate fetches point to idempotency, result reuse, and freshness rules.
  • Padding points to output contracts and unnecessary narration.
  • Repeated detours point to tool descriptions, routing, planning, memory, or missing environment signals.
Then intervene on sampled traces. Remove one candidate pattern, replay or canary the workflow, and compare task success, next-action distribution, latency, and end-to-end cost. Preserve a control group because agent runs are stochastic. A change that saves tokens but lowers completion quality has reduced consumption, not demonstrated waste removal.
Finally, fix the layer that produced the waste. The paper’s harness comparison found different inefficiencies under the same model and task. It also observed agents continuing to retry unavailable network operations when the harness did not communicate the constraint. In that situation, changing the model may be less useful than exposing capability state and terminating runs that can no longer make progress.

Why this matters to AWS builders

Amazon Bedrock publishes CloudWatch runtime metrics including  InputTokenCount ,  OutputTokenCount ,  CacheReadInputTokenCount , and  CacheWriteInputTokenCount . These metrics establish the consumption baseline and separate cached from processed input. They still need application dimensions such as workflow, task class, outcome, agent version, and harness version before a team can attribute waste.
For Agents for Amazon Bedrock, trace data can show orchestration inputs and outputs, action-group calls, knowledge-base queries, observations, failures, and token usage. That is the raw material for finding repeated fetches, unstable tool routes, and retries after an environmental failure. Teams should collect only the trace data they are authorized to retain and protect it according to its contents.
Prompt caching is a complementary control. It can reduce latency and input-token cost for supported models when requests share reusable prompt prefixes. It does not prove that the cached content is necessary, and it does not stop an agent from taking an unnecessary action. Track cache economics and trajectory waste as separate signals.
A useful operational scorecard therefore combines task success, total input and output tokens, cache reads and writes, tool-call count, retries, latency, and an intervention-validated waste estimate. This makes model choice only one part of the design discussion. Tool contracts, context policy, stop conditions, and the orchestration harness become measurable cost controls too.

Limitations and counterarguments

The research framework depends on an LLM-based judge, so some classifications remain ambiguous. Detours are especially difficult because hindsight makes reasonable exploration look unnecessary. Full reasoning traces are also unavailable for some model providers, which limits what an external audit can observe.
The reported experiments cover a finite set of models, benchmarks, and harnesses. Production workflows may have different tool semantics, compliance obligations, and costs of failure. Removing content can also alter later input context even when the immediate action remains unchanged.
For these reasons, the four patterns are diagnostic hypotheses, not automatic deletion rules. Teams should validate changes against their own task outcomes and failure costs. High-stakes workflows may rationally keep more context or verification than a token-only analysis would recommend.

Conclusion

The important distinction is between expensive work and removable work. Agents can consume many tokens because a task is difficult, while a shorter run can still waste tokens through repetition and redundant actions.
Measure complete trajectories, identify the mechanism, and verify removability before optimizing. That approach protects useful reasoning and directs engineering effort toward the model, tool interface, context policy, or harness behavior that actually caused the cost.

References

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article