AWS Builder Center

Your AI bill is rising while token prices fall. Six levers to fix it on Amazon Bedrock.

Per-token prices keep dropping, yet AI bills keep climbing. Here is how to see where your Amazon Bedrock spend goes, and six levers to cut it, in order of effort: prompt caching, batch inference, model routing, RAG tuning, semantic caching, and model customization. Last updated: November 2026. Figures reflect AWS documentation and pricing as of this date. Features, model support, and prices change often, so check the Amazon Bedrock pricing page and model cards before you plan around a number.

Leads strategic AI initiatives at AWS, helping enterprises scale agentic AI
Your AI bill is going up, and per-token prices are going down. Both are true at the same time.
Cheaper tokens make it easier to say yes to new use cases. Agents call models many times per task. Each new workload adds volume. The bill grows because the program is working, and a single "Amazon Bedrock" line on the invoice tells you nothing about which team, application, or agent is driving it.
This post covers how to fix that on AWS: first make spend visible, then apply six levers, starting with the ones that need only configuration.
With thanks to Laura Tacho and Rajdeep Banerjee at AWS for their contributions to this work.

By the numbers

Lever
What AWS reports
Prompt caching
Up to 90% lower cost on cached input tokens, and up to 85% lower latency, for supported models
Batch inference
50% lower price than on-demand, for select models
Intelligent Prompt Routing
Up to 30% lower cost without compromising accuracy
Model distillation
Up to 75% less expensive and up to 500% faster, with less than 2% accuracy loss for use cases such as RAG
Semantic caching (Amazon ElastiCache benchmark)
72.5% cost savings at a 0.90 similarity threshold

These figures come from different sources and conditions. They overlap and do not add together.

Step 0: Find out where the money goes

Every lever depends on knowing which team, application, and workload is spending tokens. Without that, you cannot tell whether a change worked. Put attribution in place before you change anything, so each change has a baseline.
Amazon Bedrock supports attribution at three levels:
1. Application inference profiles. Create one profile per workload and attach cost allocation tags such as team, cost center, and environment. Invoke models through the profile so usage and cost are recorded against those tags.
2. IAM principal attribution. Bedrock can attribute billed inference cost to the IAM identity that made the request, in AWS Cost Explorer and the Cost and Usage Report.
3. Per-request detail. Billing data is aggregated, so per-prompt detail comes from model invocation logs, where each call is its own record with its own token counts.
A gotcha for shared gateways: if every request goes through one service role, every call looks like the same caller. Have the gateway assume its role per user or per workload, or pass session tags, so attribution survives the hop. Also activate your cost allocation tags in the Billing console; tags apply from activation onward, not to past usage.
Measure outcomes, not just tokens. Cost per invocation tells you whether individual calls are efficient. Cost per successful business outcome, counting every call, retry, and tool call behind it, tells you whether the program is economical.

Lever 1: Prompt caching

Many requests resend the same long content: system instructions, tool definitions, reference documents. Prompt caching lets Bedrock reuse that repeated prefix instead of processing it again.
· Two modes. With explicit caching, you mark cache checkpoints in the prompt. With implicit caching, Bedrock reuses eligible prefixes automatically on select models such as Amazon Nova. Anthropic Claude models use explicit caching, so requests without cache controls are not cached. Check the model card for your model.
· Put static content first and dynamic content last. Order the prompt as tools, then system, then messages. If anything that changes per request sits above the checkpoint, the cache misses.
· Watch the hit rate. Bedrock publishes CacheReadInputTokenCount and CacheWriteInputTokenCount to Amazon CloudWatch. A low hit rate on a stable workload usually means the checkpoint sits after content that varies per request.
· Check the pricing. Cache reads are billed at a lower rate. On some models, cache writes cost more than standard input. Many models use a 5-minute time to live, and some support longer.

Lever 2: Batch inference

Nightly reports, document processing, and evaluation runs can wait. Bedrock batch inference runs them asynchronously at 50% of the on-demand price for select models. You submit a JSONL file in Amazon S3, and results land in S3.
Completion is not guaranteed within a fixed time, so keep anything user-facing on the real-time path. Note that prompt caching is not supported with the batch inference API, so the two levers apply to different requests.
A quick audit: list your workloads by how fast they actually need a response. Anything that can wait an hour or more is a batch candidate.

Lever 3: Route each task to the right-sized model

Only some tasks need a frontier model. Many requests are structured, formulaic, or retrieval-based, and a lighter model handles them well. Validate quality on your own workload before you shift traffic.
You can act on this in two ways:
4. Model tiering. Decide per task type which model handles it, and call that model directly.
5. Amazon Bedrock Intelligent Prompt Routing. The router predicts response quality for two models in the same family and sends each request to the better balance of quality and cost. You use the router ARN where you would use a model ID. It is optimized for English prompts, and supported models vary, so check the user guide. Check the pricing page for any routing charge on top of model usage.
Review your routing regularly. Smaller models keep improving, so a split that was right six months ago probably sends too much traffic to the expensive model today.

Lever 4: Send less retrieved context

RAG is a common source of input token growth. Amazon Bedrock Knowledge Bases default chunking uses 300-token chunks with 20% overlap, so every extra chunk adds roughly 300 tokens to every query.
· Metadata filtering narrows the search before similarity search runs.
· Hierarchical chunking matches on small chunks but returns their parents, so you retrieve fewer overlapping fragments.
· Reranking retrieves a wider candidate set and keeps the best few. A common starting point is to retrieve 20 to 50 and keep 3 to 5, then tune on your own data.
As an illustration, retrieving 10 chunks of about 300 tokens sends about 3,000 context tokens. Reranking down to the best 3 sends about 900, a 70% cut for that query. Reranking has its own per-query charge, so with a low-cost model, rerank for answer quality rather than cost.
Check what each agent receives. In multi-agent workflows, passing the full history or case file to every agent multiplies input cost by the number of agents. Give each agent only the context its step needs.

Lever 5: Semantic caching

Prompt caching reuses identical prefixes. Semantic caching goes further: when a new question means the same thing as one already answered, the application returns the stored answer and skips the model call.
Amazon ElastiCache for Valkey (version 8.2 and later) supports vector search at no additional cost beyond node pricing. In an AWS benchmark on 63,796 real chatbot queries, a 0.90 similarity threshold gave a 74.5% cache hit rate, 92.3% accuracy, and 72.5% cost savings, and average latency fell by about 72%.
· Start strict. AWS recommends a threshold of 0.90 to 0.95, lowered gradually while you monitor accuracy.
· Use a shared, persistent store. A cache that lives in each application instance misses under concurrent load, so pilot savings will not carry into production.
· Plan for stale answers. Set time to live to match how often the source content changes, and build a way to clear affected entries.

Lever 6: Model customization and distillation

Customization moves high-volume, recurring tasks to a smaller model trained for them. Bedrock supports supervised fine-tuning, reinforcement fine-tuning, and distillation. Distillation trains a smaller student model from a larger teacher model's outputs; supported pairs change over time, and distillation is not currently available for Anthropic models on Bedrock.
This pays off only on stable, high-volume workloads. Work out break-even first: a $5,000 training run that saves $400 a month breaks even around month 13. Count teacher response generation, training, and custom model storage. Refresh on a schedule, and gate each new version on an A/B comparison.

The order to pull them

Effort
Levers
What it takes
Configuration only
Prompt caching, batch inference
No new infrastructure
Architectural change
Model routing, RAG tuning
Model ID or router ARN change, retrieval changes
Infrastructure or training
Semantic caching, model customization
New infrastructure or a training run, justified by stable, high-volume workloads


Guardrails: cut cost without cutting quality

6. Baseline first. Record cost and quality before you change anything.
7. Test on real traffic, using held-out production queries rather than synthetic benchmarks.
8. One lever at a time per workload, so you can attribute the result and roll back.
9. Watch latency too. Track time to first token, so a cheaper model or tighter retrieval does not slow a user-facing flow.
10. Treat caches and logs as sensitive data. They hold real prompts. Encrypt at rest, set time to live, and isolate tenants.

Checklist: start this week

· Create a tagged application inference profile for your highest-spend workload
· Add a cache checkpoint after your static system prompt and watch the hit rate
· Move one non-urgent job to batch inference
· Check what each agent in your longest chain actually receives
· Record cost per business outcome before and after
Measure the result, then decide whether you need the next tier.

Resources

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article