Tokenomics — Understand how token consumption is calculated in Amazon Bedrock
Ever hit Bedrock throttle limits while CloudWatch shows 40% headroom? The answer is burndown rates — the hidden multiplier turning 1 output token into up to 15 quota tokens. This blog breaks down the 3-phase quota lifecycle, max_tokens optimization tricks, prompt caching impact & CloudWatch monitoring.
Introduction
When developers first encounter Amazon Bedrock throttling errors, the most common reaction is confusion: "But my CloudWatch metrics show I'm nowhere near my limit." This mismatch between observed token counts and actual quota consumption is the central puzzle that Bedrock tokenomics solves.
At its heart, tokenomics is the framework for understanding how the tokens you send and receive translate into quota consumption, billing, and the concurrency limits that govern your application's reliability. Understanding it is not optional for production workloads — it is the difference between a smoothly scaled AI application and one that mysteriously throttles at 40% of its stated quota.
This blog dissects the complete token consumption model in Amazon Bedrock, covering:
- The four token counters tracked by CloudWatch
- TPM (Tokens Per Minute) and TPD (Tokens Per Day) quota mechanics
- The burndown rate — the most misunderstood concept in Bedrock quota management
- The three-phase quota deduction lifecycle
- The critical impact of the max_tokens parameter
- Prompt caching and its effect on quota
- CloudWatch monitoring strategies
- Optimization best practices
ℹ️ Scope: All content applies to models available on thebedrock-runtimeendpoint. Models on thebedrock-mantleendpoint have separate quotas and different mechanics, noted where relevant.
Section 1 — The Four Token Counters
Amazon Bedrock tracks token consumption through four distinct CloudWatch metrics, all residing in the
AWS/Bedrock namespace. Each metric has a different role in quota calculation.Metric | What It Measures | Counts Toward Quota? |
|---|---|---|
InputTokenCount | Tokens processed by the model, excluding cached tokens | ✅ Yes (1:1) |
OutputTokenCount | Tokens generated by the model in response to a request | ✅ Yes (at burndown rate — see Section 3) |
CacheWriteInputTokens | Input tokens written to the prompt cache | ✅ Yes (1:1) |
CacheReadInputTokens | Input tokens served from the prompt cache (not reprocessed) | ❌ No — zero quota cost |
Total Effective Input Formula
A common mistake is using
InputTokenCount alone to measure your input footprint. This understates actual consumption. The correct formula is:Total effective input tokens = InputTokenCount + CacheWriteInputTokens💡 Key Insight: Cache-read tokens (CacheReadInputTokens) are served from a pre-warmed cache and are explicitly excluded from both quota calculations and billing. This is what makes prompt caching so powerful for quota efficiency — see Section 6.
Section 2 — TPM and TPD Quota Mechanics
Tokens Per Minute (TPM) — The Rolling 60-Second Window
TPM is the primary throttling gate in Amazon Bedrock. It represents the maximum number of burndown-adjusted tokens (both input and output combined) that can be consumed within any rolling 60-second window for a given model.
TPM is not an instantaneous rate — it is a count accumulated across a rolling 60-second period. This distinction matters critically for burst workloads. A batch of parallel requests may each individually appear modest, but collectively exhaust the per-minute bucket within the first few seconds, generating
ThrottlingException: Too many tokens errors even when your average daily rate is well within TPD.⚠️ Burst Warning: If your workload sends many requests simultaneously (e.g., batch processing, fan-out patterns), design for the 60-second window — not just the daily average. Spread retries across the full next 60-second cycle, not just a few seconds.
Tokens Per Day (TPD) — The Daily Cumulative Cap
TPD is the cumulative daily cap on token consumption. By default:
TPD (default) = TPM × 24 × 60 (i.e., TPM × 1,440 minutes)Both TPM and TPD are set at the model level and are adjustable via AWS Service Quotas. Each model family has its own independent TPM and TPD quota bucket.
New Account Reduced Quotas
New AWS accounts receive reduced default quotas that are lower than the standard TPM × 1,440 product. If your application hits daily limits unexpectedly early, this is the most likely cause. Request quota increases proactively via the AWS Service Quotas console before production launches.
Section 3 — Token Burndown Rates
What Is a Burndown Rate?
The burndown rate is the multiplier that converts a single generated output token into its quota-system equivalent. It is the most important and most misunderstood concept in Bedrock tokenomics.
Two critical clarifications upfront:
- It is NOT a billing multiplier. You are billed only for actual tokens used, regardless of burndown rate.
- It is exclusively a throttling/quota-management construct. It affects only how fast your TPM and TPD quota buckets are depleted.
Why Does It Exist?
Generating output tokens is computationally far more expensive than processing input tokens. Input processing is a single forward pass through the attention stack. Output generation requires an autoregressive decode step for every token produced — a sequential, GPU-memory-intensive operation that holds resources for the full duration of generation.
The burndown rate reflects this asymmetric compute cost in the quota system, ensuring that heavy generation workloads do not crowd out lighter inference workloads sharing the same regional infrastructure.
Exact Burndown Rates by Model Family
Model Family | Output Burndown Rate | Quota Impact | Input Rate |
|---|---|---|---|
Anthropic Claude v4.8 and above | 15× | 1 output token → 15 quota tokens | 1:1 |
Anthropic Claude Sonnet 5 | 10× | 1 output token → 10 quota tokens | 1:1 |
Anthropic Claude Opus 5 | 10× | 1 output token → 10 quota tokens | 1:1 |
All other Anthropic Claude (v4.7 and below) | 5× | 1 output token → 5 quota tokens | 1:1 |
All other models (Amazon Nova, Meta Llama, Mistral, etc.) | 1× | No amplification | 1:1 |
Worked Example — Claude Sonnet 4 (5× burndown)
A request with 1,000 input tokens that generates 100 output tokens:
Quota consumed = 1,000 + (100 × 5) = 1,500 tokensBilling charges = 1,000 + 100 = 1,100 tokens only🔑 Key Rule: Quota consumed ≠ Billing charges. The burndown rate amplifies quota consumption but never your bill.
bedrock-runtime vs. bedrock-mantle Scope
Burndown rates apply only to models available on the
bedrock-runtime endpoint. Models available exclusively on the bedrock-mantle endpoint have separate input and output quotas — burndown does not apply to them, and quota accounting works differently.
Section 4 — The Three-Phase Quota Deduction Lifecycle
Every Amazon Bedrock inference request moves through three distinct quota-accounting stages. Understanding all three is essential for diagnosing throttling — especially the critical Phase 1 vs. Phase 3 gap that causes the most developer confusion.
Phase 1 — Initial Reservation (The Throttle Gate)
When a request arrives, Bedrock immediately deducts a reservation from your TPM and TPD quotas before any output is generated:
Reserved = (InputTokenCount + CacheWriteInputTokens) + max_tokensThis is the only gate that determines whether the request is throttled. If your quota bucket cannot accommodate this reservation, you receive a
ThrottlingException immediately.⚠️ Critical: The burndown multiplier does not apply at Phase 1. Bedrock reservesmax_tokensraw (not burndown-adjusted) at the start, then applies the multiplier at settlement (Phase 3). This means a largemax_tokenscan cause throttling even when your actual output will be small.
Phase 2 — Dynamic Adjustment (In-Flight)
During generation, Bedrock periodically releases the portion of the reserved
max_tokens bucket that corresponds to tokens already generated. This frees capacity for concurrent requests while the current request is still running.Phase 2 is not directly observable via CloudWatch metrics. It operates as an internal optimization to improve effective concurrency.
Phase 3 — Final Settlement (End of Request)
At the end of the request, the total quota consumed is calculated and any unused reservation is replenished:
Final consumed = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)Note that
CacheReadInputTokens are explicitly excluded from this formula. Any difference between the Phase 1 reservation and the Phase 3 final value is returned to your quota bucket.💡 Why CloudWatch misleads you: TheEstimatedTPMQuotaUsagemetric reflects Phase 3 (final settled values). It can show you at 40% utilization while you're still getting throttled — because the throttle happened at Phase 1 when reservations temporarily exhausted the bucket.
Summary: The Three-Phase Lifecycle
Stage | When | Formula | Purpose |
|---|---|---|---|
1 — Reserve | Request arrives | (input tokens) + max_tokens | Throttle gate |
2 — Adjust | During generation | Incremental release of unused reservation | Free capacity for concurrency |
3 — Settle | Request completes | input + (output × burndown) | Final consumed; replenish unused |
Section 5 — The max_tokens Impact
The
max_tokens parameter is the single most impactful variable for quota efficiency. Its effect is felt exclusively at Phase 1 and has no bearing on your final billing.The Default max_tokens Danger Zone
If
max_tokens is not explicitly set, it defaults to the model's maximum output capacity. For Claude Sonnet, this is 64,000 tokens. This means:Single request reservation = input tokens + 64,000On a 2M TPM quota, a single unoptimized request reserves over 64,000 tokens at Phase 1 — meaning only ~31 concurrent requests are possible before throttling, even though the actual output may be only a few hundred tokens.
Scenario A — Oversized max_tokens (32,000)
Assume the following request parameters (5× burndown model):
InputTokenCount: 3,000CacheWriteInputTokens: 1,000CacheReadInputTokens: 4,000 (free — not counted)OutputTokenCount: 1,000max_tokens: 32,000
Phase 1 reservation: 3,000 + 1,000 + 32,000 = 36,000 tokensPhase 3 final: 3,000 + 1,000 + (1,000 × 5) = 9,000 tokensUnused replenished: 36,000 - 9,000 = 27,000 tokens❌ Problem: The 27,000 token over-reservation blocks concurrency during the entire generation window. With a 5M TPM quota, this allows only ~138 concurrent requests instead of ~555.
Scenario B — Optimized max_tokens (1,250)
Same request, but with
max_tokens right-sized to p95 expected output:Phase 1 reservation: 3,000 + 1,000 + 1,250 = 5,250 tokensPhase 3 final: 3,000 + 1,000 + (1,000 × 5) = 9,000 tokens✅ Result: Initial reservation is 5,250 vs. 9,000 final — close alignment means ~6.8× better concurrency at zero additional cost. Same tokens billed, same final quota consumed, dramatically higher throughput.
| Oversized (max_tokens = 32,000) | Optimized (max_tokens = 1,250) |
|---|---|---|
Phase 1 reservation | 36,000 tokens | 5,250 tokens |
Concurrency on 5M TPM | ~138 requests | ~952 requests |
Billing | 1,100 tokens | 1,100 tokens |
Section 6 — Prompt Caching and Quota Impact
Prompt caching allows frequently-used context (system prompts, large documents, few-shot examples) to be stored and reused across requests. It has a significant impact on both quota consumption and billing.
CacheReadInputTokens — Quota-Free
When input tokens are served from the prompt cache, they appear as
CacheReadInputTokens in CloudWatch. These tokens are completely free — they do not count toward your TPM/TPD quotas and are not billed. This is the key benefit of prompt caching for quota management.CacheWriteInputTokens — Counts Toward Quota
Tokens written to the cache on the first (or refreshed) request appear as
CacheWriteInputTokens . These do count toward your quota — they are treated identically to InputTokenCount in all quota formulas.On-Demand Formula (bedrock-runtime)
Quota consumed = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)(
CacheReadInputTokens are excluded)Provisioned Throughput Formula (Different!)
Provisioned Throughput (PT) models have a different quota formula — cache-read tokens are not free on PT, and output tokens have no burndown multiplier:
PT Quota = InputTokenCount + (CacheWriteInputTokens × 1.25) + (CacheReadInputTokens × 0.10) ← NOT free on PT + OutputTokenCount ← No burndown multiplierℹ️ PT vs. On-Demand: If you are using Provisioned Throughput, be aware that the quota mechanics differ significantly. Cache-read tokens carry a 0.10× cost (not zero), and output tokens are counted 1:1 (no burndown). Plan your PT quota accordingly.
Section 7 — Monitoring with CloudWatch
Effective token consumption monitoring requires understanding which CloudWatch metrics correspond to which phase of the quota lifecycle. Using the wrong metric leads to the classic "40% utilization, 100% throttled" puzzle.
Bedrock Automatic Dashboard
Amazon CloudWatch automatically creates a Bedrock dashboard. To access it:
- Open the CloudWatch console and select Dashboards
- Select the Automatic dashboards tab
- Select Bedrock
- In the Token Counts by Model dashboard, set your time range and observe
InputTokenCountandOutputTokenCountper model
EstimatedTPMQuotaUsage — The Phase 3 Metric
The
EstimatedTPMQuotaUsage metric (introduced in early 2026) shows your Phase 3 (settled) quota consumption as a percentage of your TPM limit. It is the most useful metric for capacity planning, but it is not what drives throttling decisions.⚠️ The Phase 1 vs. Phase 3 Gap:EstimatedTPMQuotaUsagecan show 40% while your application is actively throttling. This happens because throttles fire at Phase 1 (reservation), which can temporarily spike to 100% during burst windows even when the average settled consumption is low. Always monitor bothEstimatedTPMQuotaUsageAND yourThrottlingExceptionrate.
Model Invocation Logging for max_tokens Profiling
The
max_tokens value is a request parameter, not a CloudWatch metric. The only way to capture it for analysis is through Model Invocation Logging:- Enable Model Invocation Logging in Bedrock settings
- Logs are delivered to S3 or CloudWatch Logs
- Use CloudWatch Logs Insights or Athena to query actual
max_tokensvalues vs.OutputTokenCountto find over-reservation patterns
- Target:
max_tokensshould be within ~25% of your p95OutputTokenCountacross production traffic
Setting TPM/TPD Alarms
Metric / Alarm | Recommended Threshold / Action |
|---|---|
EstimatedTPMQuotaUsage | Alert at 70%, page at 90% — trigger quota increase request |
ThrottlingException count | Any sustained throttling → review Phase 1 reservation profile |
OutputTokenCount p95 per model | Use to right-size max_tokens per model/use-case |
InputTokenCount trend | Monitor for unexpected prompt growth; cache large stable contexts |
Section 8 — Optimization Best Practices
Apply this checklist to maximize token efficiency, reduce throttling risk, and ensure production reliability on Amazon Bedrock.
✅ 1. Right-size max_tokens via CloudWatch p95
- [ ] Enable Model Invocation Logging and analyze the
OutputTokenCountdistribution per use case
- [ ] Set
max_tokensto ~125% of your p95 output size (add a small safety margin)
- [ ] For streaming use cases,
max_tokenscan be set even tighter since Phase 2 releases tokens progressively
✅ 2. Use Prompt Caching for Repeated Large Contexts
- [ ] Identify static or slow-changing context blocks: system prompts, documentation, few-shot examples
- [ ] Enable prompt caching to convert
InputTokenCount→CacheReadInputTokens(quota-free)
- [ ] For a 10,000-token system prompt sent 1,000 times, caching saves 10M quota tokens per run
✅ 3. Choose the Right Model Burndown Tier
Scenario | Recommended Tier | Burndown | Effective TPM Gain |
|---|---|---|---|
Simple Q&A, classification | Claude Haiku / Nova Micro | 1× | 15× vs Claude v4.8 |
Complex reasoning, code | Claude Sonnet 4 (v4.7) | 5× | 3× vs Claude v4.8 |
State-of-the-art tasks | Claude v4.8+ / Opus 5 | 10–15× | Baseline |
✅ 4. Use Cross-Region Inference (CRIS) Profiles
- [ ] Enable Cross-Region Inference by using model IDs with
us.,eu., orap.prefixes
- [ ] CRIS automatically routes requests across multiple AWS regions, effectively multiplying your available TPM
- [ ] Recommended for production workloads with variable burst patterns
✅ 5. Use CountTokens API for Pre-Flight Sizing
- [ ] Call the
CountTokensAPI before sending large requests to get an exact token count at no charge
- [ ] Use this to gate oversized requests, dynamically adjust
max_tokens, or route to appropriate model tiers
- [ ] Especially useful for document processing workflows with variable input sizes
✅ 6. Batch Inference for Non-Latency-Sensitive Workloads
- [ ] Bedrock Batch Inference uses separate quota pools and offers 50% price reduction
- [ ] Ideal for overnight processing, evaluations, dataset enrichment
- [ ] Batch jobs do not compete with real-time inference for TPM quota
✅ 7. Implement Graceful Throttle Handling
- [ ] Always implement exponential backoff with jitter for
ThrottlingExceptionhandling
- [ ] Spread retries across the full 60-second TPM window, not just a few seconds
- [ ] Use a request queue with rate-limiter to smooth burst patterns before they hit Bedrock
- [ ] Pre-emptively request quota increases via AWS Service Quotas before production launches
Conclusion
Understanding Bedrock tokenomics transforms your ability to build reliable, cost-efficient AI applications. The key mental model is a four-stage lifecycle:
- Tokenize: Your text is converted into tokens — the atomic unit of all quota and billing calculations
- Reserve: Phase 1 deducts
(input tokens + max_tokens)from your TPM/TPD bucket — this is where throttling decisions are made
- Process: Phase 2 periodically releases unused reservation as tokens are generated, improving concurrency
- Settle: Phase 3 finalizes consumption as
input + (output × burndown_rate), replenishing any over-reservation
Three principles to keep front of mind:
- Quota ≠ Billing: Burndown rates amplify quota consumption for output-heavy models, but your bill is always based on actual tokens only
- max_tokens is the biggest lever: Right-sizing it is a free throughput multiplier — sometimes as high as 6–8×
- Phase 1 drives throttling, Phase 3 drives CloudWatch: Monitor both
EstimatedTPMQuotaUsageandThrottlingExceptionrate to get the complete picture
With these patterns in place, you can plan quota capacity accurately, optimize throughput without increasing costs, and build AI applications that scale reliably on Amazon Bedrock.
Author
Anshuman Nanda is a Senior Solutions Architect at Amazon Web Services. He works with startups across Middle-East and Africa. He focuses on Data Analytics and Generative/Agentic architectures, often diving deep through live demos and hands-on solutions.
References
[2] Improve operational visibility for inference workloads — New CloudWatch metrics for TTFT and Estimated Quota Consumption (AWS ML Blog, March 2026)
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article