AWS Builder Center

Amazon Bedrock: Five Things Every Startup Should Know

Essential Amazon Bedrock implementation guide for startups: API authentication strategies, Cross-Region Inference scaling, prompt caching cost optimization, and the throughput gotchas hiding in max_tokens and latency metrics. Real-world tactics, no corporate speak—learn what actually moves the needle when building production AI applications.

Solutions Architect, Startups
Written by Igor Fil (Startup Solutions Architect) and  Jean Malha  (Senior Specialist Solutions Architect - Agentic Architecture)
Mind Your Keys: Essential API Authentication Gotchas for Startups Building with Amazon Bedrock
When building your AI-powered application with Amazon Bedrock, you face a critical early decision: how to authenticate. Here's the thing though—this isn't about what stage your startup is at. It's about what your tech stack can actually support.
Start with the best option:  IAM roles with AWS STS temporary credentials . They auto-expire, eliminate static credentials entirely, and save you from managing secrets.
But sometimes you can't. Maybe you're routing requests through a custom API gateway that doesn't support IAM, or you're using a third-party orchestration layer that only speaks bearer tokens. These middleware components often expect OpenAI-style authentication and have no idea what to do with AWS credentials. For these situations, Bedrock provides  two API key types , and understanding their tradeoffs prevents costly mistakes later.
 Long-term keys  last anywhere from 1 to 36,600 days—you pick the expiration. Bedrock automatically creates an IAM user named  BedrockAPIKey-xxx , attaches the  AmazonBedrockLimitedAccess  policy, and you're off to the races. The catch? Each IAM user supports only two keys, and longer validity means longer exposure if something goes wrong.
Short-term keys are the middle ground. They expire in 12 hours max, use pre-signed URLs, and inherit permissions from whoever created them. If your app can handle refreshing tokens automatically, these give you API key simplicity with better security hygiene.
One critical detail: you cannot retrieve API keys after generation. Store them in AWS Secrets Manager immediately. Not later. Not after you test them. Immediately.
Bedrock authentication decision tree

Scale Smart: Understanding Cross-Region Inference Options for Your Bedrock Workloads
As your startup's AI application gains traction, you'll need infrastructure that scales with you. Amazon Bedrock offers  Cross-Region Inference (CRIS)  to unlock higher throughput by intelligently distributing requests across multiple Regions. However, choosing the right CRIS option matters—the wrong choice could violate compliance requirements or leave cost savings on the table. Understanding all three inference approaches helps you scale confidently while meeting your specific requirements.
By default, Bedrock uses single-Region inference: you specify a model ID like  anthropic.claude-sonnet-4-5-20250929-v1:0 , and all processing stays in that Region. This approach gives you full control over data location and supports Provisioned Throughput, making it ideal for strict data sovereignty requirements or predictable development workloads. However, you're constrained by that Region's quota limits.
When you need higher throughput but must comply with geographic data residency laws (GDPR, HIPAA, or regional banking regulations),  Geographic CRIS  routes requests across multiple Regions within the same geography. Using profile formats like  us.model-name  or  eu.model-name , Bedrock intelligently distributes your requests across Regions—for example, US requests may route between us-east-1, us-east-2, and us-west-2—while guaranteeing your data never leaves that geographic boundary.
For maximum scale without data residency constraints,  Global CRIS  delivers the highest throughput by routing to any supported commercial Region worldwide. Using the  global.model-name  format, you gain automatic global load balancing, failover capabilities, and approximately 10% cost savings on token pricing. This option suits high-scale production workloads with unpredictable global traffic patterns.
Feature
Single-Region
Geographic CRIS
Global CRIS
Throughput
Lowest
Higher
Highest
Data Location
One specific Region
Within geography (US/EU/APAC)
Any commercial Region worldwide
Cost
Standard
Standard
~10% savings
Data Residency
✅ Full control (stays in one region)
✅ Within geographic boundaries
❌ No geographic restrictions
Provisioned Throughput
✅ Supported
❌ Not supported
❌ Not supported
The model id format follows the following structure:  {geography}.{provider}.{model-name}-{version}:0 
where geographic prefixes are:
  •  us  - United States (routes between US regions: us-east-1, us-east-2, us-west-1, us-west-2, ca-central-1, ca-west-1)
  •  eu  - Europe (routes between EU regions: eu-central-1, eu-west-1, eu-west-2, eu-west-3, eu-north-1, eu-south-1, eu-south-2, il-central-1)
  •  apac  - Asia Pacific (routes between APAC regions: ap-south-1, ap-northeast-1, ap-northeast-2, ap-northeast-3, ap-southeast-1, ap-southeast-2, ap-southeast-3, ap-southeast-4, ap-south-2)
  •  global  - Global (routes to ALL commercial AWS Regions worldwide, including opt-in regions)
For example:
  • Standard single-region ID:  anthropic.claude-3-5-sonnet-20241022-v2:0  (stays in one region)
  • Geographic CRIS ID:  us.anthropic.claude-4-5-sonnet-20250929-v2:0  (routes within US geography)
  • Global CRIS ID:  global.anthropic.claude-sonnet-4-5-20250929-v1:0  (routes anywhere globally)
The routing mechanism works identically for both CRIS types: you send requests to your source Region, Bedrock evaluates model availability and current capacity across destination Regions, routes to the optimal location, and returns responses through your source Region. All CloudTrail logging happens in your source Region only, simplifying audit requirements.
One gotcha catches many teams: IAM policies for CRIS require permissions across multiple Regions. Single-Region inference needs only the model ARN in one Region. Geographic CRIS requires access to the inference profile plus the model in your source Region and all destination Regions within that geography. Global CRIS adds complexity—you must allow  "aws:RequestedRegion": "unspecified"  in your policies. Check the  supported models and Regions  documentation to configure your IAM policies correctly.
The decision framework is straightforward: use single-Region for development, strict sovereignty requirements, or when using Provisioned Throughput. Choose Geographic CRIS for compliance-sensitive production workloads in banking, healthcare, or government. Select Global CRIS when performance and cost optimization outweigh data location concerns - when using Global CRIS AWS will automatically route requests to available resources,  maximizing throughput . All three options calculate pricing from your source Region with no additional routing fees—Global CRIS simply applies an approximate 10% discount on top.
Bedrock CRIS decision

What actually moves between regions?
Aspect
Single-Region
Geographic CRIS
Global CRIS
Data storage location
Source region only
Source region only
Source region only
Input prompts (request data)
Stay in source region
Move to destination regions
Move to destination regions
Output results (responses)
Stay in source region
Generated in destination regions
Generated in destination regions
Persistence in destination
N/A
In-memory only, not stored
In-memory only, not stored
Transit encryption
N/A
Encrypted via AWS backbone
Encrypted via AWS backbone
CloudTrail logs
Source region
Source region only
Source region only
Multi-modal data (images)
Stay in source region
Base64 in prompt, not stored in destination bucket
Base64 in prompt, not stored in destination bucket
Cache Smart: Reducing Costs and Latency with Prompt Caching
As your AI application matures, you'll likely notice patterns in your prompts—system instructions, document context, tool definitions, and few-shot examples that remain constant across requests.  Prompt caching  transforms these repetitive elements into a cost and performance advantage, delivering up to 90% reduction in inference costs and 85% faster response times by storing and reusing computed prompt prefixes.
The mechanism is straightforward: you define cache checkpoints that mark contiguous sections of your prompt to cache. Subsequent requests with identical prefixes leverage cached content instead of recomputing tokens. However, getting the implementation right requires attention to detail.
Structure your prompts with static content first, followed by cache checkpoints, then dynamic content like user queries. For chat-with-document workflows, cache the document context and vary only the queries. For coding assistants, cache frequently referenced code files. For agentic workflows, cache system prompts and tool definitions that persist across multi-step tasks.  Amazon Nova models  offer automatic caching for all text prompts, while Claude models require explicit checkpoint placement—though Bedrock's simplified cache management  automatically  checks back approximately 20 content blocks from your checkpoint to find optimal matches.
One critical gotcha: cache hits require exact prefix matches. Any alteration—even whitespace changes—results in a cache miss. Cache entries have a default 5-minute TTL (Time to Live) that resets on each hit;  some models support extended 1-hour TTL  for prompts reused less frequently. Monitor your implementation using response fields like  CacheReadInputTokens  and  CacheWriteInputTokens  to track cache efficiency and validate cost savings.
Real-world results demonstrate the impact:  Care Access achieved 86% cost reduction and 66% faster processing  for medical record analysis, saving 4-8+ hours of daily processing time. For startups processing documents, running coding assistants, or building agentic applications, prompt caching represents one of the highest-impact optimizations available—implement it early to compound savings as your usage grows.
Don't Let  max_tokens  Tank Your Throughput
Here's a common trap: setting  max_tokens  to your model's maximum context window might feel like the "safe" choice, but it's actually a throughput killer. Here's why: Amazon Bedrock reserves quota at the start of each request using this formula:  input_tokens + max_tokens . For models like  Claude Sonnet 4 with a 5x quota burndown rate , an inflated  max_tokens  creates a massive initial reservation that chokes throughput. Let's say you send a request with 3,000 input tokens and set  max_tokens  to 32,000. Bedrock immediately reserves 35,000 tokens from your Tokens Per Minute (TPM) quota—even if your response only generates 1,000 output tokens. At the end of the request, only 8,000 tokens are actually consumed (3,000 input + 1,000 output × 5 burndown rate), and the unused tokens get replenished. But during processing, you've locked up 35,000 tokens, starving concurrent requests and tanking throughput. The smarter approach? Use  CloudWatch metrics  to analyze your actual  OutputTokenCount  patterns, then set  max_tokens  close to what you need. If you discover your responses average 1,200 tokens, setting  max_tokens  to 1,250 drops your initial reservation to just 4,250 tokens—over 8x better quota utilization. Make observability your friend, and watch your API performance soar.
Not All Latency Metrics Are Created Equal
Here's a subtle trap that can lead to misleading performance comparisons: different AI providers report latency differently, and understanding these differences reveals that performance might be better than it appears at first glance. Many providers prominently display Time to First Token (TTFT)—the milliseconds from request submission until the first word appears—as their primary latency metric. This makes sense for user experience since users see output faster with streaming. However,  Amazon Bedrock's default CloudWatch metrics  show InvocationLatency, which measures end-to-end time from request to complete response—not TTFT. This creates an apples-to-oranges comparison. For example, another provider might advertise "200ms latency" (their TTFT), while your Bedrock dashboard shows "2,500ms" (your E2E latency), creating a perception gap when actual performance could be more comparable than the numbers suggest. Always clarify what "latency" means when comparing providers—the real story might be more nuanced than surface-level numbers suggest, and Bedrock gives you the observability tools to measure what actually matters for your use case.
To get the full picture, start with CloudWatch  GenAI Observability dashboard  —a free, zero-configuration view that shows invocation counts, latency, token usage, and throttles across all your models. For request-level details, enable  model invocation logging  to CloudWatch Logs or S3, which captures full inputs and outputs with drilldown to specific Request IDs.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article