Continued from part 1 :

 

Set up a dashboard with these metrics per model:

Namespace: AWS/Bedrock
Dimensions: {ModelId: "\\<your-model-id>"}

1. Invocations (Sum, Period: 60s) → RPM
2. EstimatedTPMQuotaUsage (Sum, Period: 60s) → TPM utilization
3. InvocationThrottles (Sum, Period: 60s) → Failed requests
4. InvocationLatency (p99, Period: 60s) → "the time from when a request is sent to when the last token is received, end-to-end latency

CloudWatch Alarm Example

TPM warning (70% of quota) — assumes a 6M Opus 4.6 Global CRIS TPM quota; recompute 0.70 × actual_quota from servicequotas.get_service_quota (code L-3DCCFAA4):

{
"AlarmName": "Bedrock-Opus46-TPM-Warning",
"Namespace": "AWS/Bedrock",
"MetricName": "EstimatedTPMQuotaUsage",
"Dimensions": [{"Name": "ModelId", "Value": "global.anthropic.claude-opus-4-6-v1"}],
"Statistic": "Sum",
"Period": 60,
"EvaluationPeriods": 5,
"Threshold": 4200000,( 0.70 × current_quota)
"ComparisonOperator": "GreaterThanThreshold",
"TreatMissingData": "notBreaching"

}

Before any major launch or traffic increase:

· [ ] Identify peak expectations — What's your expected P90 TPM and RPM?

· [ ] Compare against current quotas — Use Service Quotas console or CLI

· [ ] Right-size max_tokens — Profile your actual output distribution

· [ ] Enable CRIS — Switch to global. prefix if not already

· [ ] Request increases early — Allow 3+ business days for processing

· [ ] Set up monitoring — CloudWatch alarms at 70% and 85%

· []Check the Adjustable column first. Some Bedrock quotas are non-adjustable fixed limits — those can't be self-service increased and need Support/account team.

· []Verify the model isn't Legacy/Deprecated. "Quota increases are not granted for models that are scheduled for retirement" — migrate to the successor instead.

· [ ] Test failover behavior — What happens when you hit the limit? Does your app retry gracefully?

Via AWS Console:

1. Navigate to Service Quotas → Amazon Bedrock

2. Find your model's TPM/RPM quota

3. Click "Request quota increase"

4. Provide use case justification

Via AWS CLI: bedrock-runtime adjustable quotas

aws service-quotas request-service-quota-increase \
--service-code bedrock \
--quota-code L-3DCCFAA4 \
--desired-value 6000000
--region us-east-1 # Service Quotas is regional; omitting it targets your default region

· Production workload with consistent, predictable traffic

· SLA requirements that can't tolerate on-demand throttling

· Running at >70% of on-demand quota consistently

· Request Reserved Tier — Contact your AWS account team (TAM, AM, or SA)

· Specify capacity needs: input TPM, output TPM, region, model, minimums — 100,000 input TPM / 10,000 output TPM, 1- or 3-month term.

· Wait for approval (2–5 business days)

· Reserved Tier is billed monthly, and billing continues until you delete the reservation via your account team

· Sizing includes cache writes: reserved TPM consumption counts InputTokenCount + CacheWriteInputTokens — if you use prompt caching, sum both when sizing the reservation.

· Dedicated CloudWatch dimensions-metrics are available under ModelId, ServiceTier, and ResolvedServiceTier dimensions.

 

Update application code to include serviceTier parameter:

response = client.converse(
modelId="global.anthropic.claude-opus-4-6-v1",
messages=messages,
serviceTier={"type": "reserved"} # ← Required to use Reserved Tier
)

Verify — Check API response and AWS CloudTrail Events to confirm requests are served from reserved capacity

Common mistake: Forgetting to pass serviceTier: { type: "reserved" }. Without this parameter, your reserved capacity sits idle while traffic hits on-demand limits.

When your application needs more tokens-per-minute capacity than what you reserved, the service automatically overflows to the Standard tier.

 

· Exponential backoff with jitter

· Circuit breaker pattern for sustained errors

· Log x-amzn-RequestId for every request (needed for support escalation)

· Monitor Invocations drop-off alongside error rates

Managing Bedrock quotas well isn't just about avoiding errors — it's about building confidence that your GenAI application can scale reliably. The teams that succeed treat capacity management as a continuous operational practice, not a one-time setup task.

If you're building RAG applications with Amazon Bedrock Knowledge Bases, there are additional quotas beyond LLM model limits that you must plan for. These are managed separately and have different default values and increase processes. It is invoked via bedrock-agent-runtime.rerank (there's also cohere.rerank-v3-5:0). Rerank is request-based, so an RPM framing is reasonable. One request carries a query plus up to 1,000 source documents.

Reranking re-scores retrieved chunks before they reach your LLM. You can call it through the Rerank API, by invoking a reranker model directly, or inside a Retrieve call using rerankingConfiguration. That last path is easy to overlook. Each Retrieve call with reranking turned on uses rerank quota.

All quotas are per account and per Region. Check current values in Service Quotas.

Key points:

· Capacity is constrained — Unlike LLM model quotas, Rerank quota increases require manual engineering review and approval timelines can extend to 1–2 weeks depending on region and capacity.

· Not self-service via Service Quotas console — Rerank increases currently go through support cases, not the standard self-service quota increase workflow.

· Region matters — Capacity varies by region. Popular regions (us-east-1, eu-central-1) may have longer approval timelines.

· Plan for scale early — If your RAG application will serve 100 concurrent users each sending 2 queries a minute, with one reranked Retrieve per query, already reaches 200 RPM at steady state. Retries and multiple Retrieve calls per turn push it higher.

 

Open a support case (Technical → Amazon Bedrock → Limits)

Specify: account ID, region, model ID (amazon.rerank-v1:0), current limit, requested limit

Include: use case description, user count (current + projected), expected peak RPM

Allow 2–3 weeks before your launch date

The Retrieve API (bedrock-agent-runtime:Retrieve) has its own separate quota:

The Retrieve quota counts API calls, not user queries. Retries and several Retrieve calls per conversation turn multiply your request rate, so measure calls per user query during load testing.

Increase process: These quotas can't be raised in the Service Quotas console, so open an AWS Support case. Include load-test data, your target rate, and your launch date. Turning on reranking adds a separate model-inference quota, so request that increase separately. Before asking for more, cut the load: cap retries, use backoff with jitter, cache repeated queries, and simplify metadata filters.

Amazon recently introduced Managed Knowledge Bases which handle retrieval and reranking internally as a managed service. If you're hitting Retrieve + Rerank quota ceilings, BMKB may provide better scalability since it manages capacity internally.

 

However:

· Migration from standalone KB Retrieve + Rerank to BMKB requires application changes

· Check that the managed option is available in your Region and supports your data source connectors.

· Run a load test in a non-production account at your expected peak, just as you would for the standard setup.

Guardrails quotas work differently from Retrieve and Rerank. They limit both the number of requests and the amount of text processed, measured in text units (1 text unit = up to 1,000 characters). Each policy you turn on has its own text-unit quota, so you can be throttled even while you're under the request limit.

Amazon Bedrock Guardrails now offers two safeguard tiers: Classic and Standard. Understanding the difference is important for both performance and quota management.

Safeguard tiers apply to:

· Content filters (text) and prompt attacks

· Denied topics

 

The policy with the lowest limit sets your throughput. If your guardrail uses content filters, denied topics, and the PII filter, the quota with the lowest limit for that traffic causes throttling first. In eu-central-1, that's usually content filters or denied topics.

Large inputs use up text units quickly. A single 50,000-character input is 50 text units, which is already the full eu-central-1 content-filter quota for one second. Checking RAG context or whole documents costs far more than checking a short user prompt.

Guardrails can run twice per turn. Checking both the input and the model output doubles the text units used.

Self-service. Unlike Retrieve and Rerank, you can request increases in the Service Quotas console. Request each policy's text-unit quota you need, not only requests per second.

If you use the standalone ApplyGuardrail API (model-agnostic guardrails), this limit applies regardless of which tier you're on.

If your existing guardrails are on Classic tier, migrating to Standard provides better detection accuracy but may behave differently for edge cases:

Test first — Deploy Standard tier in a non-production environment

Compare outputs — Run your evaluation suite against both tiers

Update guardrail configuration — Set tierConfig.tierName = "STANDARD" in both contentPolicyConfig and topicPolicyConfig, and add a crossRegionConfig guardrail profile. Check the field names against the CreateGuardrail API reference before publishing.

request increases for Standard-tier quotas, because they're separate from the Classic

 

quotas. 💡 Tip: If you're creating new guardrails, always use Standard tier

 

 

 

The thread running through all of this is the same: on Amazon Bedrock, the token or request counts you see are rarely the numbers your quota actually sees. Output tokens burn down at a multiple, max_tokens is reserved before a single token is generated, and a RAG request quietly fans out across a Retrieve quota, a reranker quota, an embedding-model quota, and — if you're guarding it — a guardrails quota, each metered and raised independently. No single dashboard captures all of that, which is why the teams that scale GenAI smoothly treat capacity as an operational practice rather than a launch-day checkbox: they right-size max_tokens, monitor the metric that reflects true consumption, size every component of the pipeline separately, and verify the real defaults for their account, Region, and resource type in the Service Quotas console before they need the headroom. Do that, and quota management stops being the thing that pages you at 2 a.m. and becomes the quiet reason your application just keeps serving.