AWS Builder Center

Tokenomics — Understand how token consumption is calculated in Amazon Bedrock

Ever hit Bedrock throttle limits while CloudWatch shows 40% headroom? The answer is burndown rates — the hidden multiplier turning 1 output token into up to 15 quota tokens. This blog breaks down the 3-phase quota lifecycle, max_tokens optimization tricks, prompt caching impact & CloudWatch monitoring.

Introduction

When developers first encounter Amazon Bedrock throttling errors, the most common reaction is confusion: "But my CloudWatch metrics show I'm nowhere near my limit." This mismatch between observed token counts and actual quota consumption is the central puzzle that Bedrock tokenomics solves.
At its heart, tokenomics is the framework for understanding how the tokens you send and receive translate into quota consumption, billing, and the concurrency limits that govern your application's reliability. Understanding it is not optional for production workloads — it is the difference between a smoothly scaled AI application and one that mysteriously throttles at 40% of its stated quota.
This blog dissects the complete token consumption model in Amazon Bedrock, covering:
  • The four token counters tracked by CloudWatch
  • TPM (Tokens Per Minute) and TPD (Tokens Per Day) quota mechanics
  • The burndown rate — the most misunderstood concept in Bedrock quota management
  • The three-phase quota deduction lifecycle
  • The critical impact of the max_tokens parameter
  • Prompt caching and its effect on quota
  • CloudWatch monitoring strategies
  • Optimization best practices
ℹ️ Scope: All content applies to models available on the  bedrock-runtime  endpoint. Models on the  bedrock-mantle  endpoint have separate quotas and different mechanics, noted where relevant.



Section 1 — The Four Token Counters

Amazon Bedrock tracks token consumption through four distinct CloudWatch metrics, all residing in the  AWS/Bedrock  namespace. Each metric has a different role in quota calculation.
Metric
What It Measures
Counts Toward Quota?
 InputTokenCount 
Tokens processed by the model, excluding cached tokens
✅ Yes (1:1)
 OutputTokenCount 
Tokens generated by the model in response to a request
✅ Yes (at burndown rate — see Section 3)
 CacheWriteInputTokens 
Input tokens written to the prompt cache
✅ Yes (1:1)
 CacheReadInputTokens 
Input tokens served from the prompt cache (not reprocessed)
❌ No — zero quota cost

Total Effective Input Formula

A common mistake is using  InputTokenCount  alone to measure your input footprint. This understates actual consumption. The correct formula is:
Total effective input tokens = InputTokenCount + CacheWriteInputTokens
💡 Key Insight: Cache-read tokens ( CacheReadInputTokens ) are served from a pre-warmed cache and are explicitly excluded from both quota calculations and billing. This is what makes prompt caching so powerful for quota efficiency — see Section 6.



Section 2 — TPM and TPD Quota Mechanics

Tokens Per Minute (TPM) — The Rolling 60-Second Window

TPM is the primary throttling gate in Amazon Bedrock. It represents the maximum number of burndown-adjusted tokens (both input and output combined) that can be consumed within any rolling 60-second window for a given model.
TPM is not an instantaneous rate — it is a count accumulated across a rolling 60-second period. This distinction matters critically for burst workloads. A batch of parallel requests may each individually appear modest, but collectively exhaust the per-minute bucket within the first few seconds, generating  ThrottlingException: Too many tokens  errors even when your average daily rate is well within TPD.
⚠️ Burst Warning: If your workload sends many requests simultaneously (e.g., batch processing, fan-out patterns), design for the 60-second window — not just the daily average. Spread retries across the full next 60-second cycle, not just a few seconds.

Tokens Per Day (TPD) — The Daily Cumulative Cap

TPD is the cumulative daily cap on token consumption. By default:
TPD (default) = TPM × 24 × 60 (i.e., TPM × 1,440 minutes)
Both TPM and TPD are set at the model level and are adjustable via AWS Service Quotas. Each model family has its own independent TPM and TPD quota bucket.

New Account Reduced Quotas

New AWS accounts receive reduced default quotas that are lower than the standard TPM × 1,440 product. If your application hits daily limits unexpectedly early, this is the most likely cause. Request quota increases proactively via the AWS Service Quotas console before production launches.



Section 3 — Token Burndown Rates

What Is a Burndown Rate?

The burndown rate is the multiplier that converts a single generated output token into its quota-system equivalent. It is the most important and most misunderstood concept in Bedrock tokenomics.
Two critical clarifications upfront:
  • It is NOT a billing multiplier. You are billed only for actual tokens used, regardless of burndown rate.
  • It is exclusively a throttling/quota-management construct. It affects only how fast your TPM and TPD quota buckets are depleted.

Why Does It Exist?

Generating output tokens is computationally far more expensive than processing input tokens. Input processing is a single forward pass through the attention stack. Output generation requires an autoregressive decode step for every token produced — a sequential, GPU-memory-intensive operation that holds resources for the full duration of generation.
The burndown rate reflects this asymmetric compute cost in the quota system, ensuring that heavy generation workloads do not crowd out lighter inference workloads sharing the same regional infrastructure.

Exact Burndown Rates by Model Family

Source: AWS Documentation, August 2026  [1] 
Model Family
Output Burndown Rate
Quota Impact
Input Rate
Anthropic Claude v4.8 and above
15×
1 output token → 15 quota tokens
1:1
Anthropic Claude Sonnet 5
10×
1 output token → 10 quota tokens
1:1
Anthropic Claude Opus 5
10×
1 output token → 10 quota tokens
1:1
All other Anthropic Claude (v4.7 and below)
5×
1 output token → 5 quota tokens
1:1
All other models (Amazon Nova, Meta Llama, Mistral, etc.)
1×
No amplification
1:1

Worked Example — Claude Sonnet 4 (5× burndown)

A request with 1,000 input tokens that generates 100 output tokens:
Quota consumed = 1,000 + (100 × 5) = 1,500 tokens
Billing charges = 1,000 + 100 = 1,100 tokens only
🔑 Key Rule: Quota consumed ≠ Billing charges. The burndown rate amplifies quota consumption but never your bill.

bedrock-runtime vs. bedrock-mantle Scope

Burndown rates apply only to models available on the  bedrock-runtime  endpoint. Models available exclusively on the  bedrock-mantle  endpoint have separate input and output quotas — burndown does not apply to them, and quota accounting works differently.



Section 4 — The Three-Phase Quota Deduction Lifecycle

Every Amazon Bedrock inference request moves through three distinct quota-accounting stages. Understanding all three is essential for diagnosing throttling — especially the critical Phase 1 vs. Phase 3 gap that causes the most developer confusion.

Phase 1 — Initial Reservation (The Throttle Gate)

When a request arrives, Bedrock immediately deducts a reservation from your TPM and TPD quotas before any output is generated:
Reserved = (InputTokenCount + CacheWriteInputTokens) + max_tokens
This is the only gate that determines whether the request is throttled. If your quota bucket cannot accommodate this reservation, you receive a  ThrottlingException  immediately.
⚠️ Critical: The burndown multiplier does not apply at Phase 1. Bedrock reserves  max_tokens  raw (not burndown-adjusted) at the start, then applies the multiplier at settlement (Phase 3). This means a large  max_tokens  can cause throttling even when your actual output will be small.

Phase 2 — Dynamic Adjustment (In-Flight)

During generation, Bedrock periodically releases the portion of the reserved  max_tokens  bucket that corresponds to tokens already generated. This frees capacity for concurrent requests while the current request is still running.
Phase 2 is not directly observable via CloudWatch metrics. It operates as an internal optimization to improve effective concurrency.

Phase 3 — Final Settlement (End of Request)

At the end of the request, the total quota consumed is calculated and any unused reservation is replenished:
Final consumed = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)
Note that  CacheReadInputTokens  are explicitly excluded from this formula. Any difference between the Phase 1 reservation and the Phase 3 final value is returned to your quota bucket.
💡 Why CloudWatch misleads you: The  EstimatedTPMQuotaUsage  metric reflects Phase 3 (final settled values). It can show you at 40% utilization while you're still getting throttled — because the throttle happened at Phase 1 when reservations temporarily exhausted the bucket.

Summary: The Three-Phase Lifecycle

Stage
When
Formula
Purpose
1 — Reserve
Request arrives
 (input tokens) + max_tokens 
Throttle gate
2 — Adjust
During generation
Incremental release of unused reservation
Free capacity for concurrency
3 — Settle
Request completes
 input + (output × burndown) 
Final consumed; replenish unused



Section 5 — The max_tokens Impact

The  max_tokens  parameter is the single most impactful variable for quota efficiency. Its effect is felt exclusively at Phase 1 and has no bearing on your final billing.

The Default max_tokens Danger Zone

If  max_tokens  is not explicitly set, it defaults to the model's maximum output capacity. For Claude Sonnet, this is 64,000 tokens. This means:
Single request reservation = input tokens + 64,000
On a 2M TPM quota, a single unoptimized request reserves over 64,000 tokens at Phase 1 — meaning only ~31 concurrent requests are possible before throttling, even though the actual output may be only a few hundred tokens.

Scenario A — Oversized max_tokens (32,000)

Assume the following request parameters (5× burndown model):
InputTokenCount: 3,000
CacheWriteInputTokens: 1,000
CacheReadInputTokens: 4,000 (free — not counted)
OutputTokenCount: 1,000
max_tokens: 32,000

Phase 1 reservation: 3,000 + 1,000 + 32,000 = 36,000 tokens
Phase 3 final: 3,000 + 1,000 + (1,000 × 5) = 9,000 tokens
Unused replenished: 36,000 - 9,000 = 27,000 tokens
❌ Problem: The 27,000 token over-reservation blocks concurrency during the entire generation window. With a 5M TPM quota, this allows only ~138 concurrent requests instead of ~555.

Scenario B — Optimized max_tokens (1,250)

Same request, but with  max_tokens  right-sized to p95 expected output:
Phase 1 reservation: 3,000 + 1,000 + 1,250 = 5,250 tokens
Phase 3 final: 3,000 + 1,000 + (1,000 × 5) = 9,000 tokens
✅ Result: Initial reservation is 5,250 vs. 9,000 final — close alignment means ~6.8× better concurrency at zero additional cost. Same tokens billed, same final quota consumed, dramatically higher throughput.

Oversized (max_tokens = 32,000)
Optimized (max_tokens = 1,250)
Phase 1 reservation
36,000 tokens
5,250 tokens
Concurrency on 5M TPM
~138 requests
~952 requests
Billing
1,100 tokens
1,100 tokens



Section 6 — Prompt Caching and Quota Impact

Prompt caching allows frequently-used context (system prompts, large documents, few-shot examples) to be stored and reused across requests. It has a significant impact on both quota consumption and billing.

CacheReadInputTokens — Quota-Free

When input tokens are served from the prompt cache, they appear as  CacheReadInputTokens  in CloudWatch. These tokens are completely free — they do not count toward your TPM/TPD quotas and are not billed. This is the key benefit of prompt caching for quota management.

CacheWriteInputTokens — Counts Toward Quota

Tokens written to the cache on the first (or refreshed) request appear as  CacheWriteInputTokens . These do count toward your quota — they are treated identically to  InputTokenCount  in all quota formulas.

On-Demand Formula (bedrock-runtime)

Quota consumed = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)
( CacheReadInputTokens  are excluded)

Provisioned Throughput Formula (Different!)

Provisioned Throughput (PT) models have a different quota formula — cache-read tokens are not free on PT, and output tokens have no burndown multiplier:
PT Quota = InputTokenCount
+ (CacheWriteInputTokens × 1.25)
+ (CacheReadInputTokens × 0.10) ← NOT free on PT
+ OutputTokenCount ← No burndown multiplier
ℹ️ PT vs. On-Demand: If you are using Provisioned Throughput, be aware that the quota mechanics differ significantly. Cache-read tokens carry a 0.10× cost (not zero), and output tokens are counted 1:1 (no burndown). Plan your PT quota accordingly.



Section 7 — Monitoring with CloudWatch

Effective token consumption monitoring requires understanding which CloudWatch metrics correspond to which phase of the quota lifecycle. Using the wrong metric leads to the classic "40% utilization, 100% throttled" puzzle.

Bedrock Automatic Dashboard

Amazon CloudWatch automatically creates a Bedrock dashboard. To access it:
  1. Open the CloudWatch console and select Dashboards
  1. Select the Automatic dashboards tab
  1. Select Bedrock
  1. In the Token Counts by Model dashboard, set your time range and observe  InputTokenCount  and  OutputTokenCount  per model

EstimatedTPMQuotaUsage — The Phase 3 Metric

The  EstimatedTPMQuotaUsage  metric (introduced in early 2026) shows your Phase 3 (settled) quota consumption as a percentage of your TPM limit. It is the most useful metric for capacity planning, but it is not what drives throttling decisions.
⚠️ The Phase 1 vs. Phase 3 Gap:  EstimatedTPMQuotaUsage  can show 40% while your application is actively throttling. This happens because throttles fire at Phase 1 (reservation), which can temporarily spike to 100% during burst windows even when the average settled consumption is low. Always monitor both  EstimatedTPMQuotaUsage  AND your  ThrottlingException  rate.

Model Invocation Logging for max_tokens Profiling

The  max_tokens  value is a request parameter, not a CloudWatch metric. The only way to capture it for analysis is through Model Invocation Logging:
  • Enable Model Invocation Logging in Bedrock settings
  • Logs are delivered to S3 or CloudWatch Logs
  • Use CloudWatch Logs Insights or Athena to query actual  max_tokens  values vs.  OutputTokenCount  to find over-reservation patterns
  • Target:  max_tokens  should be within ~25% of your p95  OutputTokenCount  across production traffic

Setting TPM/TPD Alarms

Metric / Alarm
Recommended Threshold / Action
 EstimatedTPMQuotaUsage 
Alert at 70%, page at 90% — trigger quota increase request
 ThrottlingException  count
Any sustained throttling → review Phase 1 reservation profile
 OutputTokenCount  p95 per model
Use to right-size  max_tokens  per model/use-case
 InputTokenCount  trend
Monitor for unexpected prompt growth; cache large stable contexts



Section 8 — Optimization Best Practices

Apply this checklist to maximize token efficiency, reduce throttling risk, and ensure production reliability on Amazon Bedrock.

✅ 1. Right-size max_tokens via CloudWatch p95

  • [ ] Enable Model Invocation Logging and analyze the  OutputTokenCount  distribution per use case
  • [ ] Set  max_tokens  to ~125% of your p95 output size (add a small safety margin)
  • [ ] For streaming use cases,  max_tokens  can be set even tighter since Phase 2 releases tokens progressively

✅ 2. Use Prompt Caching for Repeated Large Contexts

  • [ ] Identify static or slow-changing context blocks: system prompts, documentation, few-shot examples
  • [ ] Enable prompt caching to convert  InputTokenCount  →  CacheReadInputTokens  (quota-free)
  • [ ] For a 10,000-token system prompt sent 1,000 times, caching saves 10M quota tokens per run

✅ 3. Choose the Right Model Burndown Tier

Scenario
Recommended Tier
Burndown
Effective TPM Gain
Simple Q&A, classification
Claude Haiku / Nova Micro
1×
15× vs Claude v4.8
Complex reasoning, code
Claude Sonnet 4 (v4.7)
5×
3× vs Claude v4.8
State-of-the-art tasks
Claude v4.8+ / Opus 5
10–15×
Baseline

✅ 4. Use Cross-Region Inference (CRIS) Profiles

  • [ ] Enable Cross-Region Inference by using model IDs with  us. ,  eu. , or  ap.  prefixes
  • [ ] CRIS automatically routes requests across multiple AWS regions, effectively multiplying your available TPM
  • [ ] Recommended for production workloads with variable burst patterns

✅ 5. Use CountTokens API for Pre-Flight Sizing

  • [ ] Call the  CountTokens  API before sending large requests to get an exact token count at no charge
  • [ ] Use this to gate oversized requests, dynamically adjust  max_tokens , or route to appropriate model tiers
  • [ ] Especially useful for document processing workflows with variable input sizes

✅ 6. Batch Inference for Non-Latency-Sensitive Workloads

  • [ ] Bedrock Batch Inference uses separate quota pools and offers 50% price reduction
  • [ ] Ideal for overnight processing, evaluations, dataset enrichment
  • [ ] Batch jobs do not compete with real-time inference for TPM quota

✅ 7. Implement Graceful Throttle Handling

  • [ ] Always implement exponential backoff with jitter for  ThrottlingException  handling
  • [ ] Spread retries across the full 60-second TPM window, not just a few seconds
  • [ ] Use a request queue with rate-limiter to smooth burst patterns before they hit Bedrock
  • [ ] Pre-emptively request quota increases via AWS Service Quotas before production launches



Conclusion

Understanding Bedrock tokenomics transforms your ability to build reliable, cost-efficient AI applications. The key mental model is a four-stage lifecycle:
  1. Tokenize: Your text is converted into tokens — the atomic unit of all quota and billing calculations
  1. Reserve: Phase 1 deducts  (input tokens + max_tokens)  from your TPM/TPD bucket — this is where throttling decisions are made
  1. Process: Phase 2 periodically releases unused reservation as tokens are generated, improving concurrency
  1. Settle: Phase 3 finalizes consumption as  input + (output × burndown_rate) , replenishing any over-reservation
Three principles to keep front of mind:
  • Quota ≠ Billing: Burndown rates amplify quota consumption for output-heavy models, but your bill is always based on actual tokens only
  • max_tokens is the biggest lever: Right-sizing it is a free throughput multiplier — sometimes as high as 6–8×
  • Phase 1 drives throttling, Phase 3 drives CloudWatch: Monitor both  EstimatedTPMQuotaUsage  and  ThrottlingException  rate to get the complete picture
With these patterns in place, you can plan quota capacity accurately, optimize throughput without increasing costs, and build AI applications that scale reliably on Amazon Bedrock.



Author

Anshuman Nanda is a Senior Solutions Architect at Amazon Web Services. He works with startups across Middle-East and Africa. He focuses on Data Analytics and Generative/Agentic architectures, often diving deep through live demos and hands-on solutions.



References

[1]  How tokens are counted in Amazon Bedrock   (Official AWS Documentation, August 2026)
[4]  Amazon Bedrock Runtime Metrics — CloudWatch   (Official AWS Documentation)
[5]  Amazon Bedrock Quotas   (Official AWS Documentation)
[6]  Count Tokens API in Amazon Bedrock   (Official AWS Documentation)
[7]  Prompt Caching in Amazon Bedrock   (Official AWS Documentation)
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article