💸 Your AI Agent Is More Expensive Than You Think

One AI agent request can trigger multiple model calls, tool calls, and thousands of tokens. See where the hidden costs come from and how to cut them with better retrieval, routing, caching, and loop control.

Nasim Raj Laskar

I like ML

Published Sep 22, 2026

Last modified Sep 22, 2026

Your agent answered one question. It made 3 model calls, retrieved 12 chunks, called 2 tools, and reprocessed the same system prompt every time.The user sees one answer. Your AWS bill sees everything. 💀
An agent isn't priced like a chatbot. 1 user request ≠ 1 model inference.
﻿
The hidden pipeline
﻿
﻿
Where the tokens goOne question. The agent retrieves 12 chunks (~9,600 tokens) and needs 3 LLM calls to reason, call a tool, and answer. Most agent frameworks resend everything — retrieved context, system prompt, tool defs — on every call:
﻿
38,900 input tokens for one question. Bad retrieval isn't just an accuracy problem — it multiplies with every loop.
﻿
What that costs, in real dollarsReal Claude Sonnet 5 Bedrock pricing: $2/1M input, $10/1M output.
Before — 38,900 input tokens (the 3-call chain above) + ~3,000 output tokens (final answer included):
text
38,900 × $2 / 1M   = $0.0778
 3,000 × $10 / 1M  = $0.0300
                     ────────
                     $0.108
﻿
After — tighter retrieval, one fewer call: 14,000 input + 1,500 output tokens:
text
14,000 × $2 / 1M   = $0.0280
 1,500 × $10 / 1M  = $0.0150
                     ────────
                     $0.043
﻿
Same task, 60% cheaper. At 50,000 requests/day (30-day month), that's the difference between ~$161,700 and ~$64,500 — for the exact same feature:
﻿
﻿
The tool-call trapA tool call isn't just its own fee — its result becomes more input tokens on the next call:
text
User → LLM → Tool → Result → LLM (re-reads everything) → Answer
﻿
Real cost = tool fee + tool-result tokens + another model invocation.
﻿
Five fixes that actually workRetrieve less, rerank harder — 20 chunks in, 4–6 sent to the model. Cheaper and better grounded.
Route small questions to small models — Bedrock Intelligent Prompt Routing can cut costs by up to 30% (official); AWS reported 56% in one internal Anthropic-family test that matched the stronger model's response quality.
Cache what doesn't change — system prompts, tool defs, reference docs. Bedrock prompt caching: up to 90% lower inference cost and 85% lower latency for supported workloads. GPT-5.6 on Bedrock bills cached input at a flat 90% discount.
Cap the loop —  max_iterations ,  max_tool_calls ,  max_output_tokens ,  timeout . Unlimited reasoning is an unlimited bill.
Trace the request, not just the endpoint — LLM calls, input/output tokens, cache hit rate, and cost per request, not just per second.
﻿
This isn't theoreticalAWS's GenAIIC rebuilt two production agents with Works Human Intelligence — caching, tighter prompts, better model routing. Reported result: up to 97% cost reduction, same task.
﻿
These are stacked techniques on real workloads, not one universal switch — but the direction holds: most of the waste isn't the model. It's how many times you make it work.
﻿
The cost equationtext
AI COST ≈  Σ(input tokens × input price)
         + Σ(output tokens × output price)
         + tool/API costs
         + infrastructure costs
﻿
And the multiplier is the loop itself:
text
More agent loops → more LLM calls → more tokens processed → 💸 higher cost
﻿
Every term above can repeat inside one request:
text
                 ┌─ RAG
                 ├─ Tools
USER → AGENT → LLM ── Memory
                 ├─ Retries
                 └─ Loops
﻿
Before you swap models to cut costs, trace the whole request first. The savings are usually in the loop, not the model card.
Don't optimize the model before you understand the request.
﻿
Cost example uses live Claude Sonnet 5 Bedrock pricing with the stated token assumptions (38,900 in / 3,000 out → 14,000 in / 1,500 out). Token counts are an illustrative but realistic 3-call agent trace. Percentages are as reported by AWS in the linked posts.
 Prompt caching  ·  GPT-5.6 caching  ·  Intelligent Prompt Routing  ·  AgentCore case study 