Cross-Region Inference and Canadian Data Residency: A Technical Architecture Guide
Learn how Amazon Bedrock cross-region inference handles data across its three states — at rest, in transit, and in processing — and explore the questions Canadian organizations should discuss with their legal and compliance teams when evaluating this capability.
Disclaimer: This post provides technical architecture guidance to help customers understand how Amazon Bedrock cross-Region inference service (CRIS) works and what questions to explore with their legal and compliance teams. It does not constitute legal or compliance advice. Customers are responsible for evaluating their own legal, regulatory, and contractual obligations. We recommend engaging qualified Canadian privacy counsel for your specific organizational context.
Why This Matters Right Now
Several foundation models popular with Canadian organizations have recently reached end-of-life on Amazon Bedrock — Claude 3 Sonnet (July 2025), Claude 3.5 Sonnet (March 2026), and Claude 3.7 Sonnet (April 2026). Newer models such as Claude Sonnet 4.x and Haiku 4.5 are currently accessible from ca-central-1 only through CRIS.
This creates a practical challenge. Organizations running production workloads in ca-central-1 need access to current-generation models, but their legal and compliance teams may have concerns about data leaving Canada — even transiently.
In our experience working with Canadian customers across financial services, public sector, and healthcare, many organizations apply data residency requirements more broadly than their specific legal, regulatory, or contractual obligations may require. This isn't a criticism — legal and compliance teams provide guidance based on the information available to them. Our goal is to help provide the technical details they need to make fully informed recommendations.
This post walks through: - How CRIS works technically — what data moves and what stays - The three states of data and how CRIS relates to each - Questions to explore with your legal and compliance team - Architecture patterns that strengthen your compliance posture - Alternatives for workloads where CRIS is genuinely not appropriate
Understanding the Three States of Data
Data residency refers generally to a requirement to make sure certain data remains within a particular jurisdiction. However, data residency requirements can differ depending on the state of the data. Understanding these states is critical when evaluating CRIS:
| State | Description | Example | Persistence |
|---|---|---|---|
| At rest | Data stored on disk, in databases, backups, or durable storage | Customer records in a database, S3 objects, knowledge base embeddings | Indefinite — exists until explicitly deleted |
| In transit | Data moving between systems over a network | API request from your application to a service endpoint | Transient — exists only during transmission |
| In processing | Data being computed upon in volatile memory (RAM/VRAM) | Model inference — prompt loaded into GPU memory, result generated, memory freed | Ephemeral — milliseconds to seconds, then gone |
How CRIS relates to these states:
Data at rest — Data at rest remains exclusively in your source Region (ca-central-1). Logs, knowledge bases, model artifacts, configurations, CloudTrail records — none of this leaves Canada.
Data in transit — During CRIS, prompts and responses transit the AWS private backbone encrypted with TLS 1.2+. They do not traverse the public internet.
Data in processing — Inference computation occurs in volatile memory in the destination Region. The input is processed, the output is generated, and the memory is freed. No persistent copy is created when using most models. For certain models however, data is retained for offline processing in the destination Region for abuse detection. See Amazon Bedrock abuse detection for more information on which models require storage.
Organizations with data residency obligations should determine — ideally with the guidance of their legal and compliance teams — which states of data their specific requirements cover. Many requirements focus primarily on data at rest. Others may extend to data in transit or processing. This distinction is crucial when evaluating whether CRIS is appropriate for a given workload.
How CRIS Actually Works — The Technical Mechanics
Amazon Bedrock cross-Region inference operates through inference profiles — configurations that define a foundation model and the set of AWS Regions where requests can be routed. Here's what actually happens:
Your application calls the Bedrock API from ca-central-1
CRIS prioritizes the source Region when capacity exists — minimizing latency
When local capacity is insufficient, the request routes dynamically to a destination Region
The model performs inference computation entirely in volatile memory
The response returns encrypted to the source Region
Nothing persists in the destination Region — no logs, no copies, no stored outputs. 1. Note: certain models retain data temporarily for abuse detection (details ).
What Stays vs. What Moves
| Data Type | Location | Moves During Inference? |
|---|---|---|
| Customer-managed logs (model invocation logging) | Source Region only | ❌ No |
| Knowledge bases, embeddings, indexes | Source Region only | ❌ No |
| CloudTrail audit logs | Source Region only | ❌ No |
| CloudWatch metrics | Source Region only | ❌ No |
| Fine-tuned model artifacts | Source Region only | ❌ No |
| Guardrails configurations | Source Region only | ❌ No |
| Input prompts | Encrypted transit — ephemeral | ⚡ Transiently (not persisted)* |
| Output results | Encrypted transit — ephemeral | ⚡ Transiently (not persisted) * |
*Note: For certain models, data may be retained temporarily in the destination Region for abuse detection purposes. See Amazon Bedrock abuse detection for details on which models this applies to. Additionally, if you enable optional features such as model invocation logging or the Response API, data will be persisted as configured by those features.
Security Properties
Private AWS backbone only — "All data transmitted during cross-Region operations remains on the AWS network and does not traverse the public internet" (AWS Documentation – Cross-Region Inference )
End-to-end TLS 1.2+ encryption on inter-region communication
Model providers have zero access — AWS performs a deep copy of models into AWS-controlled deployment accounts. Anthropic, Meta, Mistral — none of them can access your prompts or completions, in any Region (AWS – Data Protection )
No training on customer data — AWS does not use your prompts or completions to train models, and does not share them with third-party model providers.
Logging and persistence are opt-in — Amazon Bedrock does not log customer prompts/completions unless you choose to use a feature that persists that data (e.g., Response API, invocation logging). Review abuse detection requirements for model-specific exceptions.
How This Differs from Traditional Cross-Border Data Transfer
| Dimension | Traditional Cross-Border Storage | CRIS |
|---|---|---|
| Data persistence in destination | ✅ Stored at rest | ❌ Ephemeral for most models (see abuse detection exceptions ) |
| Copy of data exists elsewhere | ✅ Full persistent copy | ❌ Transient in-memory |
| Network path | May traverse public internet | Private AWS backbone only |
| Duration in destination | Indefinite | Milliseconds to seconds* |
| Logs in destination | ✅ Often created locally | ❌ Logs stay in source Region |
| Model provider access | Varies | ❌ Providers do not have access by default* |
*Note: For certain models, data may be retained temporarily in the destination Region for abuse detection purposes. See Amazon Bedrock abuse detection for details on which models this applies to. Additionally, if you enable optional features such as model invocation logging or the Response API, data will be persisted as configured by those features.
Questions to Explore with Your Legal and Compliance Team
Based on our experience working with Canadian organizations — particularly in financial services — we've found the following framework helps structure productive conversations with legal and compliance teams.
Question 1: Do you have a legal, regulatory, or contractual obligation to keep data in Canada?
This is the gating question. The answer determines the subsequent path:
If YES — you have a specific obligation: - Which obligation specifically? (Contract clause, regulatory guideline, policy directive?) - Does it apply to data at rest, data in transit, data in processing, or all three? - Does it cover all data types, or specific categories (PII, health records, financial records)? - Can the obligation be satisfied through technical controls (encryption, anonymization, audit logging)?
If NO — but stakeholders still have concerns: - This typically becomes a data protection / security conversation rather than a regulatory one - The focus shifts to demonstrating that CRIS provides equivalent or superior security to in-region inference - The technical properties of CRIS (private backbone, encryption, no persistence, - no provider access by default) directly address these concerns
Question 2: What specific regulatory frameworks apply?
For Canadian organizations, the relevant frameworks may include:
| Framework | Applies To | Key Consideration |
|---|---|---|
| PIPEDA | Private sector, federally regulated industries | Accountability principle governs cross-border transfers. Does not prohibit them — requires comparable protection. ( OPC Guidelines ) |
| OSFI B-10 | FRFIs (banks, trust companies, insurance companies) | Third-party risk management. "Place of records" requirements in the Bank Act / Insurance Companies Act. Risk-based approach. |
| OSFI B-13 | FRFIs | Technology and cyber risk management. Risk-based, no geographic mandate. Requires documented risk assessment. ( BLG – OSFI B-13 ) |
| Quebec Law 25 | Organizations handling Quebec residents' PI | Requires Privacy Impact Assessment before cross-border transfers. Procedural requirement. ( Fasken – Bill 64 ) |
| CCCS Medium Profile | Government of Canada departments | While GC policies around Protected B focus primarily on data at rest, departmental CIOs retain discretion over processing location decisions. We recommend engaging your departmental security authority for specific guidance. |
| Provincial privacy acts (Alberta PIPA, BC PIPA, Ontario PHIPA) | Province-specific | Vary in requirements. Generally use accountability models similar to PIPEDA. |
Important notes:- OSFI B-13 addresses technology and cyber risk management broadly — it does not directly impose data residency requirements. The "place of records" provisions relevant to data location are found in B-10 and in the governing statutes (Bank Act, Insurance Companies Act, Trust and Loan Companies Act). - Many FRFIs adopt Canada-first cloud architectures as a conservative risk management posture. This is often a business decision that reflects prudent risk management rather than a specific regulatory mandate.
Question 3: For customers with CLOUD Act concerns
The US CLOUD Act is a common concern raised in these discussions. This is a nuanced legal topic that goes beyond the scope of this technical post. For customers with CLOUD Act concerns, we recommend: - Consulting the AWS Compliance page on CLOUD Act - Engaging AWS Security Assurance for a detailed discussion specific to your situation - Reviewing the AWS Data Processing Addendum provisions
The Key Architectural Facts for Your Compliance Discussion
Regardless of which framework applies, these technical facts are relevant to any assessment with one caveat that is for certain models, data may be retained temporarily in the destination Region for abuse detection purposes. See Amazon Bedrock abuse detection for details on which models this applies to.
- No data is stored outside your source Region— data at rest remains in ca-central-1
- Processing is ephemeral — volatile memory only, freed after computation
- Transit is encrypted — TLS 1.2+ on a private network
- Full audit trail in-region — CloudTrail logs every request, including which region processed it
- No third-party access — model providers cannot access your data in other regions
- Geographic profiles available — you can constrain routing to specific geographies (e.g., US only, not global)
Architecture Patterns — Defense-in-Depth
For organizations that want to strengthen their compliance posture further — whether because their obligations require it or because they prefer a belt-and-suspenders approach — Amazon Bedrock Guardrails provides additional controls.
Bedrock Guardrails can detect and anonymize sensitive information before prompts are sent for inference. This is not a requirement for CRIS to be compliance-aligned — it's a defense-in-depth pattern that helps organizations meet their accountability obligations:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import boto3
client = boto3.client('bedrock', region_name='ca-central-1')
response = client.create_guardrail(
name='canadian-data-residency-guardrail',
description='Anonymizes Canadian PII as defense-in-depth for cross-region inference',
sensitiveInformationPolicyConfig={
'piiEntitiesConfig': [
{'type': 'CA_SOCIAL_INSURANCE_NUMBER', 'action': 'ANONYMIZE'},
{'type': 'CA_HEALTH_NUMBER', 'action': 'ANONYMIZE'},
{'type': 'NAME', 'action': 'ANONYMIZE'},
{'type': 'EMAIL', 'action': 'ANONYMIZE'},
{'type': 'PHONE', 'action': 'ANONYMIZE'},
{'type': 'ADDRESS', 'action': 'ANONYMIZE'},
{'type': 'CREDIT_DEBIT_CARD_NUMBER', 'action': 'BLOCK'},
{'type': 'INTERNATIONAL_BANK_ACCOUNT_NUMBER', 'action': 'ANONYMIZE'},
{'type': 'DRIVER_ID', 'action': 'ANONYMIZE'},
],
'regexesConfig': [
{
'name': 'CA_POSTAL_CODE',
'pattern': r'[A-Za-z]\d[A-Za-z]\s?\d[A-Za-z]\d',
'action': 'ANONYMIZE',
'description': 'Canadian postal code format'
}
]
},
blockedInputMessaging='Input blocked: contains sensitive Canadian PII.',
blockedOutputsMessaging='Output blocked: response contained sensitive information.'
)ApplyGuardrail API as Independent Pre-Filter
Use ApplyGuardrail independently — before sending data to any model — to make sure sensitive content is masked regardless of the inference path (AWS Documentation – ApplyGuardrail API ):
1
2
3
4
5
6
7
8
9
10
11
12
runtime_client = boto3.client('bedrock-runtime', region_name='ca-central-1')
response = runtime_client.apply_guardrail(
guardrailIdentifier=guardrail_id,
guardrailVersion=guardrail_version,
source='INPUT',
content=[{
'text': {'text': 'Summarize the account for John Smith, SIN 123-456-789, at 45 King St W, Toronto ON M5H 1J8'}
}]
)
if response['action'] == 'GUARDRAIL_INTERVENED':
sanitized_text = response['outputs'][0]['text']
# Result: "Summarize the account for {NAME}, SIN {CA_SOCIAL_INSURANCE_NUMBER}, at {ADDRESS}"The guardrail executes in ca-central-1. PII detection and masking happen entirely within Canada. The PII-to-token mapping does not leave the source region.
Defense-in-Depth Layers
| Layer | Control | Purpose |
|---|---|---|
| 1. Network | AWS PrivateLink + VPC endpoints | Traffic does not touch public internet |
| 2. Encryption | TLS 1.2+ in transit, KMS at rest | Data encrypted between nodes |
| 3. Access | IAM least-privilege + SCPs | Organizational enforcement of geographic boundaries |
| 4. Content | Bedrock Guardrails PII anonymization | Additional protection for sensitive data elements |
| 5. Audit | CloudTrail + inferenceRegion field | Complete audit trail in source Region |
| 6. Governance | PIA documentation, risk assessments | Regulatory evidence package |
SCP to Constrain CRIS to Geographic Profiles
For organizations that want to limit cross-Region inference to specific geographies:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "DenyGlobalCRIS",
"Effect": "Deny",
"Action": "bedrock:InvokeModel*",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:RequestedRegion": "unspecified"
}
}
}]
}This allows geographic inference profiles (e.g., routing within the US only) while blocking global profiles.
When CRIS May Not Be Appropriate
Here are some examples of scenarios where CRIS may not be appropriate::
- Government Classified Data (Secret/Top Secret) — requires accredited environments with physical security controls beyond what commercial cloud provides
- Explicit Contractual Prohibitions — if a contract states "all processing within Canada" without distinguishing data states, CRIS may conflict with those commitments regardless of regulatory permissibility
- Departmental CIO Directives — for GC workloads, departmental CIOs may exercise discretion to require in-region processing even where policy permits flexibility
- Specific Regulatory Orders — Privacy Commissioner orders, consent decrees, or sector-specific frameworks that mandate Canadian processing
- Workloads Where Anonymization Isn't Feasible — situations where complete records are essential to the task and cannot be de-identified
For these cases, one path forward is self-hosting open-weight models in ca-central-1. See the Appendix below.
The Non-Binary Approach
The decision doesn't have to be all-or-nothing. Organizations can — and often do — use CRIS for workloads where it's appropriate (code generation, summarization of de-identified text, policy drafting) while keeping sensitive workloads on in-region inference or self-hosted models. Apply the appropriate control level to each data classification.
Appendix: Self-Hosting Open-Weight Models in ca-central-1
For customers with legitimate constraints — classified workloads, explicit contractual prohibitions, or organizational policies that cannot be amended — self-hosting open-weight models on Amazon SageMaker in ca-central-1 provides a viable path. Data does not leave the region, you control the endpoint lifecycle, and modern open-weight models rival or exceed Claude 3 Sonnet in capability.
Model Comparison
| Model | Parameters | License | Context | Key Strengths | vs. Claude 3 Sonnet |
|---|---|---|---|---|---|
| Llama 3.3 70B Instruct | 70B | Meta Community License | 128K | General reasoning, coding, multilingual | Exceeds on MMLU (~86% vs ~79%) |
| Mistral Large 2 | 123B | Apache 2.0 | 128K | Multilingual (FR/EN), function calling | Exceeds across the board |
| Qwen 2.5 72B | 72B | Apache 2.0 | 128K | Math, coding, CJK languages | Comparable or better |
| GPT-OSS 20B | 20B | Apache 2.0 | 128K | Reasoning, coding, efficient deployment | Lighter-weight — strong for cost-sensitive workloads |
(Artificial Analysis – Llama 3.3 vs Claude 3 Sonnet ; AWS Blog – Llama 3.3 70B on SageMaker JumpStart )
Recommendation by Use Case
| Use Case | Recommended Open-Weight Alternative |
|---|---|
| General chat & QA | Llama 3.3 70B Instruct |
| Code generation & review | Llama 3.3 70B or Qwen 2.5 72B |
| Multilingual (French/English) | Mistral Large 2 or Llama 3.3 70B |
| RAG with structured output | Mistral Large 2 (strong function calling) |
| Cost-sensitive / lighter workloads | GPT-OSS 20B (fits on single g5.2xlarge with quantization) |
| Mathematical reasoning | Qwen 2.5 72B |
GPU Instance Sizing for ca-central-1
| Instance | GPUs | VRAM | Suitable For | Hourly Cost (approx.) |
|---|---|---|---|---|
| ml.g5.2xlarge | 1× A10G | 24 GB | GPT-OSS 20B with 4-bit quantization | ~$1.52/hr |
| ml.g5.12xlarge | 4× A10G | 96 GB | 70B models with 4-bit quantization | ~$7.09/hr |
| ml.g5.48xlarge | 8× A10G | 192 GB | 70B FP16 or 123B quantized | ~$22.69/hr |
| ml.p4d.24xlarge | 8× A100 | 320 GB | Any model, full precision | ~$37.69/hr |
G5 instances are available in ca-central-1. Practical note: capacity pools are smaller than US regions — request quota increases via Service Quotas well in advance.
Deployment: GPT-OSS 20B with vLLM in ca-central-1
GPT-OSS 20B is OpenAI's open-weight model (Apache 2.0) with strong reasoning and coding capabilities. At 20B parameters, it fits on a single ml.g5.2xlarge with 4-bit quantization — the most cost-effective deployment option:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import boto3
import sagemaker
from sagemaker.model import Model
region = "ca-central-1"
session = sagemaker.Session(boto_session=boto3.Session(region_name=region))
role = sagemaker.get_execution_role()
# DJL vLLM inference container
djl_image_uri = f"763104351884.dkr.ecr.{region}.amazonaws.com/djl-inference:0.31.0-vllm0.7.3-cu124"
env = {
"HF_MODEL_ID": "openai/gpt-oss-20b",
"TENSOR_PARALLEL_DEGREE": "1",
"OPTION_ROLLING_BATCH": "vllm",
"OPTION_MAX_MODEL_LEN": "16384",
"OPTION_DTYPE": "auto",
"OPTION_QUANTIZE": "bitsandbytes",
"OPTION_GPU_MEMORY_UTILIZATION": "0.85",
}
model = Model(image_uri=djl_image_uri, role=role, env=env, sagemaker_session=session)
predictor = model.deploy(
initial_instance_count=1,
instance_type="ml.g5.2xlarge",
endpoint_name="gpt-oss-20b-ca-central",
container_startup_health_check_timeout=900
)Prompt Migration: Bedrock Advanced Prompt Optimization
Amazon Bedrock Advanced Prompt Optimization — available in ca-central-1 — automatically rewrites prompts optimized for one model to work on another (AWS Blog – Advanced Prompt Optimization ).
How it works:
- You provide: prompt templates, example inputs, ground truth outputs, evaluation metric
- The system: sends prompts to target models, evaluates responses, rewrites in a feedback loop
- You receive: optimized prompt templates with scores, cost estimates, latency comparisons
1
2
3
4
5
6
7
8
9
10
11
client = boto3.client('bedrock', region_name='ca-central-1')
response = client.create_advanced_prompt_optimization_job(
jobName='claude-to-llama-migration',
modelConfigurations=[
{'modelId': 'anthropic.claude-3-sonnet-20240229-v1:0'}, # Baseline
{'modelId': 'meta.llama3-3-70b-instruct-v1:0'}, # Target
],
inputConfig={'s3Uri': 's3://my-bucket/prompt-optimization/input/'},
outputConfig={'s3Uri': 's3://my-bucket/prompt-optimization/output/'}
)The critical point: this runs entirely in ca-central-1. Your prompts, test data, and optimization results do not leave the region.
Cost Comparison
| Deployment | Monthly (24/7) | Per 1M Tokens (est.) |
|---|---|---|
| Claude 3 Sonnet (Bedrock, pay-per-token) | Variable | $3.00 input / $15.00 output |
| Llama 3.3 70B (g5.12xlarge, AWQ) | ~$5,105 | ~$0.30–0.50 at moderate load |
| GPT-OSS 20B (g5.2xlarge, 4-bit) | ~$1,094 | ~$0.10–0.20 at moderate load |
Self-hosting becomes cost-effective at sustained volumes exceeding ~500K tokens/hour — common for production workloads.
Conclusion
Canadian organizations facing model deprecation have options. The technical architecture of CRIS — ephemeral processing, no data persistence outside the source Region (except for certain models where data may be retained temporarily in the destination Region for abuse detection purposes ), encrypted transit on a private backbone — provides a fundamentally different risk profile than traditional cross-border data transfer.
We encourage organizations to:
- Verify your specific obligations — work with your legal and compliance team to determine exactly what your requirements cover (data at rest? in transit? in processing?)
- Understand the technical facts — this post provides the architecture details that inform a proper risk assessment
- Consider defense-in-depth — Guardrails, SCPs, encryption, and audit logging strengthen compliance posture
- Know your alternatives — for genuinely constrained workloads, self-hosted open-weight models in ca-central-1 keep everything in-region
The path forward isn't binary. Organizations can use CRIS where appropriate while keeping restricted workloads on in-region solutions. The key is making that determination based on a complete understanding of both the technical architecture and your specific obligations.
All regulatory references should be validated with qualified Canadian privacy counsel for your specific use case and organizational context. AWS provides compliance resources through AWS Artifact and AWS Security Assurance for detailed discussions.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article