Amazon Bedrock: Architecture for Reliability and Resilience

High Throughput and Multi-Region Failover with CRIS. Building Resilient AI Workloads with Amazon Bedrock Cross-Region Inference

Sara

Published Jun 28, 2026

Last modified Jul 7, 2026

Introduction

As generative AI workloads on Amazon Bedrock grow in scale, managing throughput efficiently becomes essential — especially when on-demand capacity reaches its regional limits during peak traffic. Cross-Region Inference (CRIS) addresses this by automatically distributing requests across multiple AWS Regions seamlessly — with minimal code changes and no complex load-balancing logic.

This post walks through setting up CRIS, explains how Global CRIS works, and shows how to build failover using multiple source regions for complete resilience.

Key takeaway: CRIS handles throttling automatically. For source region non availability, application-level failover provides complete resilience — covered in Part 3.

Part 1: Setting Up CRIS

Step 1 — Find Your Inference Profile

CRIS uses inference profiles — pre-defined resources that map a model to a set of destination regions. You use the profile ID instead of the model ID.

1
2
3
4
5
6
7
8
9
import boto3

bedrock = boto3.client("bedrock", region_name="us-east-1")

response = bedrock.list_inference_profiles()

for profile in response["inferenceProfileSummaries"]:
    print(f"{profile['inferenceProfileName']} — {profile['inferenceProfileId']}")
    print(f"  Type: {profile['type']}")

Step 2 — Configure IAM Permissions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": [
        "arn:aws:bedrock:*::foundation-model/*",
        "arn:aws:bedrock:*:ACCOUNT_ID:inference-profile/*"
      ]
    }
  ]
}

If your organization uses SCPs with region restrictions, allowlist all destination regions in the CRIS pool.

Step 3 — Invoke with CRIS

The only code change — replace the model ID with the inference profile ID:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import boto3

import json



bedrock_runtime = boto3.client("bedrock-runtime", region_name="us-east-1")



# Use the CRIS inference profile ID instead of direct model ID

inference_profile_id = "us.anthropic.claude-sonnet-4-20250514-v1:0"



response = bedrock_runtime.converse(

    modelId=inference_profile_id,

    messages=[

        {

            "role": "user",

            "content": [{"text": "Explain cross-region inference in one paragraph."}]

        }

    ],

    inferenceConfig={

        "maxTokens": 512,

        "temperature": 0.7

    }

)



output_text = response["output"]["message"]["content"][0]["text"]

print(output_text)

Bedrock routes this request to whichever US region has available capacity. No additional configuration needed.

Part 2: How Global CRIS Works

Geographic CRIS routes within a boundary (US, EU, APAC). Global CRIS removes that boundary — routing to any supported commercial region worldwide for maximum throughput.

Geographic vs. Global

Type	Profile Prefix	Routes To	Use When
Geographic (US)	`us.`	us-east-1, us-east-2, us-west-2	Data must stay in the US
Geographic (EU)	`eu.`	Supported EU commercial regions	GDPR / EU residency
Geographic (APAC)	`apac.`	Supported APAC regions	APAC residency
Global	model ID (no geo prefix)	All 30+ commercial regions	Maximum throughput

How Global Routing Decides

When you invoke a Global CRIS profile, Bedrock evaluates in this order:

Try the source region first (where you called from)
If capacity is constrained, route to the nearest region with availability
Optimize for lowest latency among available destinations

All routing is transparent. Your logs, billing, and CloudTrail events remain in the source region. Data is encrypted in transit across Amazon's private network.

Performance with CRIS

CRIS dramatically improves success rates and reduces latency compared to single-region endpoints under load. Workloads that previously experienced throttling during peak traffic achieve consistent, reliable responses once CRIS distributes requests across the destination region pool.

Source Region is Implicit

Your source region is wherever your client connects. Bedrock automatically recognizes it as the origin for routing. Every supported region with access to the profile serves as a valid source, enabling multiple application instances across regions to leverage the same CRIS profile in parallel.

1
2
3
4
5
# Source region = us-east-1
client = boto3.client("bedrock-runtime", region_name="us-east-1")

# Source region = ap-southeast-1
client = boto3.client("bedrock-runtime", region_name="ap-southeast-1")

Supported Source Regions for Global CRIS

Global CRIS supports 20+ source regions including regions where models are accessible exclusively through CRIS (such as India, South Africa, Jakarta, and Thailand). If ListInferenceProfiles returns the profile from a given region, that region is a valid source.

Part 3: Failover Using Different CRIS Source Regions

CRIS distributes inference compute across regions automatically. For source region unavailability, application-level failover provides complete resilience. The approach: maintain multiple source regions and switch between them when one becomes unreachable.

Client-Side Failover Logic

Your application maintains clients for two source regions and fails over on connection errors:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
import boto3
from botocore.exceptions import (
    EndpointConnectionError,
    ClientError,
    ConnectionClosedError
)

class BedrockCRISFailover:
    def __init__(self, primary="us-east-1", secondary="us-west-2"):
        self.regions = [primary, secondary]
        self.clients = {
            r: boto3.client("bedrock-runtime", region_name=r)
            for r in self.regions
        }
        # Same CRIS profile works from any source region in the geo
        self.profile_id = "us.anthropic.claude-sonnet-4-5-20250929-v1:0"

    def invoke(self, messages, max_tokens=512, temperature=0.7):
        last_error = None

        for region in self.regions:
            try:
                response = self.clients[region].converse(
                    modelId=self.profile_id,
                    messages=messages,
                    inferenceConfig={
                        "maxTokens": max_tokens,
                        "temperature": temperature
                    }
                )
                return {
                    "region": region,
                    "output": response["output"]["message"]["content"][0]["text"],
                    "failover": region != self.regions[0]
                }

            except (EndpointConnectionError, ConnectionClosedError) as e:
                last_error = e
                continue

            except ClientError as e:
                code = e.response["Error"]["Code"]
                if code in ("ServiceUnavailableException", "InternalServerError"):
                    last_error = e
                    continue
                raise

        raise RuntimeError(f"All regions failed. Last error: {last_error}")

# Usage
client = BedrockCRISFailover(primary="us-east-1", secondary="us-west-2")

result = client.invoke(
    messages=[{"role": "user", "content": [{"text": "What is Amazon Bedrock?"}]}]
)

print(f"Served from: {result['region']} | Failover: {result['failover']}")

Architecture Options for API-Based Failover

For production APIs fronted by Lambda, here are three architecture patterns:

Option 1 — Route 53 Failover Routing

Route 53 health checks monitor each API Gateway endpoint. When the primary returns unhealthy, DNS automatically resolves to the secondary region. Failover time depends on TTL and health check interval.

Key Advantages

Simple DNS Failover (TTL ~60s). Low Cost. Works with any endpoint. Configurable health checks

Option 2 — CloudFront Origin Failover

CloudFront origin groups enable per-request failover. If the primary origin returns 5xx or times out, CloudFront immediately retries the failover origin within the same request — with no DNS propagation delay.

Key Advantages

Layer 7 routing. Per request failover (Instant). Edge caching. Native WAF Integration. No Static IPs needed.

Option 3 — Global Accelerator for Bedrock Failover

For Bedrock workloads requiring the fastest possible failover, Global Accelerator provides health-check-based traffic shifting at the network layer. Each endpoint group contains an ALB in a different region, backed by Lambda invoking Bedrock with CRIS. When health checks detect that a region's Bedrock endpoint is unreachable, traffic automatically shifts to the healthy endpoint group within approximately 30 seconds.

This pattern is ideal for Bedrock workloads because:

Faster failover than DNS — Traffic shifts at the network layer (~30s) rather than waiting for DNS TTL expiration
Consistent entry point — Static anycast IPs mean clients always connect to the same address regardless of which region is active
Health checks validate the full chain — The ALB health check can probe the Lambda→Bedrock path end-to-end, ensuring traffic only flows to regions where Bedrock CRIS is responding successfully

Key Advantages

Layer 4 Routing. Static IPs (no client reconfigs). ~30s failover. TCP/UDP layer routing. Built-in DDoS protection.

Which Architecture to Choose

Pattern	Failover Speed	Complexity	Best For
Route 53	60–120s (DNS TTL)	Low	Most Bedrock workloads
CloudFront	Per-request (instant)	Medium	Latency-sensitive inference APIs
Global Accelerator	~30s (health check)	Medium	Static IP requirements, fastest automated failover

Summary

Problem	Solution
Throttling (429 errors)	CRIS — automatic, requires only a profile ID swap
Source region unavailability	Application-level failover across source regions
Data residency + throughput	Geographic CRIS (US/EU/APAC)
Maximum global throughput	Global CRIS

CRIS eliminates throttling with a single code change. For complete resilience, pair it with multi-source-region failover at the application layer — giving your workloads both high throughput and high availability.

References

# amazon-bedrock # resilience # availability # disaster-recovery

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Amazon Bedrock: Architecture for Reliability and Resilience

Sara

Introduction

Part 1: Setting Up CRIS

Step 1 — Find Your Inference Profile

Step 2 — Configure IAM Permissions

Step 3 — Invoke with CRIS

Part 2: How Global CRIS Works

Geographic vs. Global

How Global Routing Decides

Performance with CRIS

Source Region is Implicit

Supported Source Regions for Global CRIS

Part 3: Failover Using Different CRIS Source Regions

Client-Side Failover Logic

Architecture Options for API-Based Failover

Option 1 — Route 53 Failover Routing

Option 2 — CloudFront Origin Failover

Option 3 — Global Accelerator for Bedrock Failover

Which Architecture to Choose

Summary

References