AWS Builder Center
Amazon Bedrock Service Tiers: Priority vs Flex

Amazon Bedrock Service Tiers: Priority vs Flex

Compare Amazon Bedrock Priority, Default, and Flex tiers with real benchmarks for latency, cost, and workload-aware routing.

The Problem: Every Request Gets the Same Tier

When I first saw Bedrock's new service tiers, I thought the decision was going to be pretty straightforward: use Priority when latency matters, Default for normal traffic, and Flex for anything that can wait.
But before putting that logic into a router, I wanted to answer a simpler question: does Priority actually make my requests faster?
The pricing table makes the case seem obvious — Priority costs roughly 1.75× Default, Flex costs roughly 0.5× Default. The implication baked into that pricing is that you get something proportional in return. I didn't want to build a router on top of that assumption without testing it first. So I tested it.
The results changed how I thought about the whole problem.



Why This Experiment, Why Now

AWS documentation tells you what each tier is intended for. Pricing tells you the obvious economic incentive. But neither tells you what happens with your specific workload, region, and traffic pattern.
I didn't want to build another "Flex is for batch, Priority is for real-time" demo. I wanted to see what actually happened when I ran real requests through all three tiers, modeled the economics across different traffic distributions, and built a routing layer I could actually read, reason about, and change without guessing at the consequences.
That is what led to this project: a measurable, reproducible experiment rather than a confident assumption presented as a tutorial.



Architecture: Four Small Modules, One Clean Pipeline

I intentionally kept the router boring.
No LangChain. No LLM classifier. No abstraction layers. Just four small Python modules doing one thing each: validate the workload label, choose a tier, check whether that tier is available, and return a decision. I wanted to look at any routing outcome and immediately understand why it happened.
classify → policy → capability check → routing decision
Overview Architecture - for Amazon Bedrock Service Tier Router

Component responsibilities:
Module
Role
 router/classifier.py 
Validates and normalizes workload category strings
 router/policy.py 
Maps workload categories to preferred service tiers
 router/capability.py 
Checks tier availability; selects deterministic fallbacks
 router/router.py 
Orchestrates classify → policy → capability → decision
Why four modules instead of one function? I split the concerns because the policy is the part I expect to change most often. By isolating it in a frozen dataclass, I can swap in a different policy — database-backed, A/B split, per-tenant — without touching the capability checker or the orchestrator.
AWS services used:
  • Amazon Bedrock Runtime ( bedrock-runtime ) — the  converse  API with the  serviceTier  parameter. This is how you specify which tier to use; omitting it defaults to Default.
  • boto3 — the AWS SDK for Python.
Why not a framework? The routing logic is a pure decision function. Adding LangChain or a similar framework would add indirection without adding value. The router is a component that plugs into your existing Bedrock call, not a replacement for it.



Implementation Walkthrough

Step 1: Validate the workload category

I wanted invalid workload labels to fail early rather than silently becoming a potentially expensive routing decision.
# router/classifier.py

class WorkloadCategory(str, Enum):
CRITICAL = "critical"
INTERACTIVE = "interactive"
BACKGROUND = "background"

def classify_workload(workload_category: str | WorkloadCategory) -> WorkloadCategory:
if workload_category is None:
raise InvalidWorkloadCategoryError("Workload category is required.")

if isinstance(workload_category, WorkloadCategory):
return workload_category

normalized = workload_category.strip().lower()

try:
return WorkloadCategory(normalized)
except ValueError as exc:
valid = ", ".join(c.value for c in WorkloadCategory)
raise InvalidWorkloadCategoryError(
f"Unknown workload category: {workload_category!r}. "
f"Expected one of: {valid}."
) from exc
The  str  enum base class means a  WorkloadCategory  behaves like a string in comparisons and CSV output — no  .value  everywhere.

Step 2: Apply the routing policy

Keeping the policy in its own module was deliberate. I expect this to be the part I change most often as I learn more about the actual workload.
# router/policy.py

@dataclass(frozen=True)
class RoutingPolicy:
mapping: dict[WorkloadCategory, ServiceTier]

@classmethod
def default(cls) -> "RoutingPolicy":
return cls(
mapping={
WorkloadCategory.CRITICAL: ServiceTier.PRIORITY,
WorkloadCategory.INTERACTIVE: ServiceTier.DEFAULT,
WorkloadCategory.BACKGROUND: ServiceTier.FLEX,
}
)

def preferred_tier(self, workload: WorkloadCategory) -> ServiceTier:
try:
return self.mapping[workload]
except KeyError as exc:
raise UnsupportedPolicyError(
f"No routing policy exists for workload: {workload!r}."
) from exc

Step 3: Check capability and resolve fallbacks

This ended up being more important than I initially expected. Choosing a tier is one thing; actually being able to use it is another. The capability checker resolves to an available fallback using a configured chain:
# router/capability.py (fallback defaults)
# priority → default
# flex → default
# default → no fallback (RuntimeError if unavailable)

def resolve(self, requested_tier: str | ServiceTier) -> tuple[ServiceTier, bool]:
requested = self.normalize_tier(requested_tier)

if self.is_available(requested):
return requested, False # (resolved_tier, used_fallback)

fallback = self._fallback_order.get(requested)

if fallback is not None and self.is_available(fallback):
return fallback, True

raise RuntimeError(
f"Requested tier '{requested.value}' is unavailable and no "
"available fallback is configured."
)

Step 4: Compose in the Router

At this point the router is intentionally simple: it turns a workload label into a concrete tier decision.
# router/router.py

class Router:
def __init__(
self,
policy: RoutingPolicy | None = None,
capability_checker: TierCapabilityChecker | None = None,
) -> None:
self.policy = policy or RoutingPolicy.default()
self.capability_checker = capability_checker or TierCapabilityChecker()

def route(self, workload_category: str | WorkloadCategory) -> RoutingDecision:
workload = classify_workload(workload_category)
requested_tier = self.policy.preferred_tier(workload)
resolved_tier, used_fallback = self.capability_checker.resolve(requested_tier)

return RoutingDecision(
workload=workload,
requested_tier=requested_tier,
resolved_tier=resolved_tier,
used_fallback=used_fallback,
)

Step 5: Pass the resolved tier to Bedrock

This is the part that makes the project more than a routing abstraction — the decision ultimately becomes a real  serviceTier  parameter in a live Bedrock call.
decision = router.route("background") # → RoutingDecision(resolved_tier=FLEX, ...)

response = bedrock.converse(
modelId="apac.amazon.nova-pro-v1:0",
messages=[{"role": "user", "content": [{"text": prompt}]}],
inferenceConfig={"maxTokens": 128, "temperature": 0},
serviceTier={"type": decision.resolved_tier.value}, # ← this is the key param
)
The  serviceTier  parameter was added to the  converse  API alongside the tier launch. If you are on an older boto3 version (pre-1.34), it will not exist — upgrade to at least  boto3==1.43 .
IAM permissions required — no new permissions beyond standard model invocation:
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel"],
"Resource": "arn:aws:bedrock:ap-southeast-1::foundation-model/amazon.nova-pro-v1:0"
}
Tier selection is a parameter in the existing  converse  call, not a separate API.



The Gotcha: Two Things I Expected That Turned Out to Be Wrong

Gotcha 1: Priority did not demonstrate lower latency in the experiment

This was the assumption I was most confident about going in: Priority should be faster. The documentation describes it as the fastest tier and says it prioritizes requests over Default and Flex. I fully expected to see that in the data.
The controlled tier experiment ran 300 requests across all three tiers in randomized order. Here is what came back:
Tier
Mean (ms)
P95 (ms)
Priority
~803
~1,080
Default
~733
~1,045
Flex
~711
~903
Honestly, I was surprised. The documentation says Priority is the fastest tier, but the experiment didn't show a statistically significant latency advantage for it.
I don't think those two things are actually contradictory — I think the experiment just didn't create conditions where the difference becomes visible. The requests were intentionally tiny: "Explain Amazon SQS in exactly three concise sentences." That is roughly 10 input tokens and ~58 output tokens per request, running at 20 RPM under light load. There is no real demand contention for Priority to resolve in your favor under those conditions.
My hypothesis is that the difference would become more observable with larger prompts and at higher request rates, where tiers would actually compete for inference capacity. I haven't tested that yet, so I won't claim it — but it is the next experiment I would run.
For now, if I had to explain Priority tier to a stakeholder, I would frame it this way: you are paying for prioritized access, not raw speed. That distinction matters more when there is actual demand competing for capacity. For short, low-frequency requests, you are unlikely to feel it.

Gotcha 2: Workload-aware routing can cost more, not less

My second assumption was even simpler: if Flex is cheaper, sending batch workloads there should obviously save money.
It does — but only if the rest of your workload does not contain too much Priority traffic.
Here is the break-even rule I derived from the observed token profile:
~1.48 Flex-routed background requests are required to offset the cost of one Priority-routed critical request.
The policy sensitivity model confirmed the consequence. Of 231 simulated workload mixes, only 96 (42%) produced a cheaper result than All-Default. 134 (58%) were more expensive. The model makes the extremes concrete:
Workload mix
Savings vs. All-Default
0% critical / 100% background
+50.00%
100% critical / 0% background
−75.00%
If your batch workload is genuinely deferrable, Flex is a compelling default. But I wouldn't route something to Flex just because it's called "batch." The real question is how much latency and availability variance you are willing to trade for the lower cost. "Batch" tells you how the job runs; it doesn't tell you how tolerant that job is to degraded service. Those are different questions, and the routing decision should answer the second one.



Results: What the 100-Request Benchmark Actually Showed

The benchmark ran 100 live Bedrock requests in  ap-southeast-1  against  apac.amazon.nova-pro-v1:0 :
  • 20 critical → Priority
  • 50 interactive → Default
  • 30 background → Flex

Routing

This was the part I most wanted to verify: did the router actually do what I told it to do?
Metric
Result
Policy mapping accuracy
100.00%
Tier resolution accuracy
100.00%
Request success rate
100.00%
Fallback rate
0.00%
In this run, yes. Every workload resolved to the intended tier, every request succeeded, and the fallback path was never needed.

Cost

Metric
Value
Estimated router cost
$0.019490
All-Default baseline
$0.019523
Savings
$0.000034 (0.17%)
This was smaller than I expected. The Priority premium on 20 critical requests almost exactly cancelled the Flex savings on 30 background requests. The takeaway is not that the router fails — it is that workload composition matters more than simply "using tiers." The router gives you the lever; your traffic distribution determines whether that lever saves money or costs you more.

Token usage

The benchmark deliberately used a small, standardized prompt so the tier comparison stayed controlled: 1,000 input tokens and 5,851 output tokens across 100 requests, averaging about 10 input and 58 output tokens per call. That also means the result should not be generalized to large-context workloads.



Lessons Learned: What I Would Do Differently

1. Randomize request order from the start

Running critical → interactive → background in contiguous blocks means workload category is correlated with request time. Any time-varying service behavior contaminates the comparison. In the next experiment, I randomize.

2. Don't leave classification until the end

I built the router around an explicit label first because it made the experiment cleaner. But that exposed a gap: in a real application, something still has to decide whether a request is critical, interactive, or background. That problem is non-trivial, and I would solve it before calling this production-ready. A lightweight heuristic, or a Haiku-class model classifying on request metadata, is probably the next piece.

3. Instrument the workload mix before you ship

This became obvious only after I saw the 0.17% result. The economic value of this router is entirely determined by your critical-to-background ratio. Without a dashboard showing that ratio in real time, you have no idea whether the router is saving or costing money on any given day. Add the metric before the feature, not after.

4. Stress-test Flex before relying on it

Flex is the only tier with an explicit throttling caveat in the AWS documentation. I tested it at 20 RPM under light load and saw zero fallbacks. That is not a stress test. Before routing anything important to Flex, verify how it behaves at or above expected peak load.

5. Re-verify pricing per region before any billing claim

The pricing table I used reflects  ap-southeast-1  at project time. Cross-region inference profiles, other regions, and tier pricing can all differ. Any cost claim in production needs current figures from the AWS pricing page for the exact model and configuration.

6. Test different prompt sizes

This is probably the biggest open question I have after finishing the experiment. Tiny prompts are good for controlled comparison but limit how much the latency result generalizes. A follow-up should stratify by token count to test whether tier differences become more observable as input and output sizes grow — which is also where the cost differences become more meaningful.



What This Is, and What It Isn't

After running the experiments, this is the claim I am comfortable making:
A deterministic workload-aware router can successfully map critical, interactive, and background workloads to Priority, Default, and Flex service tiers respectively — with 100% routing accuracy in a 100-request benchmark, an explicit fallback mechanism, and a measurable economic model whose value is determined by workload distribution.
It is not a claim that the router universally saves money, that Priority is reliably faster, or that this policy is production-optimal. Those claims would require more evidence than one benchmark run against one model in one region at light load.
I started this project thinking the tier decision would be straightforward. The experiments made that mental model considerably less simple.
Priority didn't produce the latency advantage I expected under small-prompt conditions. Flex can reduce cost, but only when enough of the workload can safely tolerate it. And the economics can flip quickly once Priority traffic becomes a large share of the mix.
That is ultimately why I think the router is useful — not because it magically makes Bedrock cheaper, but because it forces the tier decision to become explicit, measurable, and testable. Before building this, I was making implicit assumptions. The router turned those assumptions into a policy I can read, change, and validate against data.



Get the Code

The full project — router, benchmark scripts, analysis, and results — is on GitHub:  bedrock-tier-router 
To run the unit tests locally (no AWS credentials needed):
git clone https://github.com/<your-org>/bedrock-tier-router.git
cd bedrock-tier-router
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pytest -v
Questions, corrections, and production war stories about Bedrock tier behavior are welcome in the repo issues or in the AWS Community Builders Slack.


Kayne Rodrigo — AWS Community Builder, AI/ML Engineering Track



Appendix: Pricing Assumptions

These are the analytical rates used in all cost calculations in this post. Re-verify against current AWS pricing before using for any billing estimate.
Tier
Input / 1M tokens
Output / 1M tokens
Default
$0.80
$3.20
Priority
$1.40
$5.60
Flex
$0.40
$1.60
Model:  apac.amazon.nova-pro-v1:0 , Region:  ap-southeast-1 , cross-region inference profile.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article