AWS Builder Center

Keep Your AI Inside the Fence: A Controlled-Access RAG Architecture on AWS

Most RAG breaches aren't jailbreaks — they're permission bugs. Here's how to enforce access control at the retrieval layer on AWS, before any data reaches your LLM.

Keep Your AI Inside the Fence: A Controlled-Access RAG Architecture on AWS




Introduction

Generative AI is transforming how enterprises surface and act on internal knowledge. Retrieval-Augmented Generation (RAG) — grounding an LLM's answers in your own documents — sits at the center of this shift. But as teams race to ship RAG-powered applications, one question usually gets answered too late:
Who decides what the AI is allowed to know?
A model that can retrieve anything your data store holds, regardless of who's asking, isn't just a compliance risk — it's a breach waiting to happen. Sensitive HR files, financial projections, unreleased product roadmaps: if your RAG pipeline treats all of these as equally fetchable, you've handed your AI a master key.
This post walks through a Controlled-Access RAG architecture on AWS — a design that enforces data boundaries at every layer of the pipeline, so the AI only ever surfaces what the requesting user is actually authorized to see.



The Problem with Default RAG

A typical out-of-the-box RAG flow looks like this:
Query → Embed → Vector Search (top-K) → Stuff into Prompt → LLM Answer

Every step here runs on semantic similarity alone. There's no check for who is asking. A junior analyst and a CFO asking the identical question get identical retrieval results — including any confidential chunk that happens to score high on relevance.
That's not an edge case. It's the default behavior of nearly every off-the-shelf RAG pipeline, because vector search was never designed to be an authorization system.



The Fix: Push Access Control Into the Retrieval Layer

Instead of trusting the model to "behave," the architecture below enforces the boundary before anything reaches the LLM. Four checkpoints do the work:
  1. Identity-Aware Gateway — verify who's asking before the request goes anywhere.
  1. Metadata-Tagged Vector Store — every chunk carries its own access rules.
  1. Pre-Filtered Retrieval — vector search only ever looks inside the chunks the user is permitted to read.
  1. Audit Logging — every retrieval is recorded: who, what, and which chunks came back.

End-to-end flow

┌───────────────────────┐
User Request ───────▶│ Amazon Cognito │ Authenticates user,
│ (Identity Layer) │ issues JWT with roles
└───────────┬────────────┘
│ verified identity + claims
▼
┌───────────────────────┐
│ API Gateway + │ Validates token,
│ Lambda Authorizer │ attaches user context
└───────────┬────────────┘
│ authorized request + user context
▼
┌───────────────────────┐
│ RAG Orchestrator │ Builds a filtered
│ (Lambda) │ query from user context
└───────────┬────────────┘
│ query + access filter
▼
┌───────────────────────┐
│ Amazon OpenSearch │ kNN search, restricted
│ (Vector Store) │ to permitted chunks only
└───────────┬────────────┘
│ permitted chunks only
▼
┌───────────────────────┐
│ Amazon Bedrock │ Generates the answer
│ (Claude / Titan) │ from authorized context
└───────────┬────────────┘
│
▼
┌───────────────────────┐
│ CloudWatch + S3 │ Full audit trail:
│ Audit Logs │ who, query, chunks used
└───────────────────────┘

Notice where the enforcement actually happens: at the OpenSearch query itself, not after the fact. Chunks the user can't see are never scored, never ranked, and never handed to the model.



Layer 1 & 2: Identity and Authorization, Before Anything Else

Every request carries a signed JWT from Amazon Cognito, containing the user's groups, department, and clearance level. API Gateway, backed by a Lambda Authorizer, validates that token and turns it into a user context object — the thing every downstream service will use to decide what's visible.
JWT ──▶ Lambda Authorizer ──▶ { userId, groups, department, clearanceLevel }
│
└── attached to every downstream call

No anonymous queries. No shared service accounts touching the vector store directly. If the token doesn't validate, the request never reaches retrieval.



Layer 3: Metadata Travels With the Data

The fence is actually built at ingestion time, not query time. As documents are chunked and embedded, each chunk is tagged with its own access rules — classification, permitted departments, permitted groups, owner.
Document ─▶ Chunk ─▶ Embed ─┐
├─▶ {embedding, text, access_metadata} ─▶ OpenSearch
Access Policy (from source)─┘

This matters because it means access control isn't a separate lookup table that can silently drift out of sync with reality — it's baked directly into the record being searched.



Layer 4: Filtering Happens During Retrieval, Not After

This is the load-bearing wall of the whole design. Amazon OpenSearch Service supports combining vector (kNN) search with metadata filters in a single query — so the similarity search itself only ever runs over chunks the user is cleared to see.
User Context ──▶ permitted classifications + groups
│
▼
┌─────────────────────────────────┐
│ kNN search AND metadata filter │ ──▶ Top-K permitted chunks
└─────────────────────────────────┘

Compare that to the common (and risky) shortcut: fetch the top-50 results, then filter out anything unauthorized in application code. That approach still lets sensitive content influence rankings and can leak information through timing or partial results. Filtering inside the query avoids that entirely — unauthorized chunks are never retrieved in the first place.
# The one line that matters most in this whole architecture:
# the filter rides alongside the kNN clause, not after it.
query = {
"query": {"bool": {
"must": [{"knn": {"embedding": {"vector": query_embedding, "k": 5}}}],
"filter": [{"terms": {"access_metadata.permitted_groups": user_groups}}]
}}
}




Layer 5: Generation With Amazon Bedrock

Only the authorized chunks reach this stage. They're assembled into a prompt and sent to Bedrock (Claude, Titan, etc.) for generation.
Permitted Chunks ─▶ Prompt Assembly ─▶ Bedrock (LLM) ─▶ Grounded Answer

A well-written system prompt ("answer only from the provided context") is still good practice — but it's a soft guardrail. The hard guardrail already happened one layer up: the model physically never received anything it wasn't supposed to see.



Layer 6: Every Retrieval Leaves a Trail

Each query produces an audit record — user identity, department, clearance, which chunk IDs were returned — streamed to CloudWatch for real-time monitoring and archived to S3 for compliance.
Retrieval Event ─▶ CloudWatch Logs (real-time)
└─▶ S3 Archive (long-term / compliance)

Store a hash of the query rather than the raw text where possible — you want traceability without creating a second copy of sensitive user input sitting in your logs.



IAM: The Backstop

Everything above assumes your compute itself is scoped tightly. The Lambda functions running this pipeline should hold least-privilege roles — read access to OpenSearch, invoke access to a specific Bedrock model, write access to a specific log group. No wildcard actions, no wildcard resources. If an application-layer control ever fails, IAM is what keeps the blast radius small.



Key Design Principles, Summarized

Principle
Implementation
Identity before retrieval
Cognito JWT + Lambda Authorizer
Policy travels with the data
Access metadata attached at ingestion
Filter during retrieval, not after
OpenSearch metadata filter alongside kNN
Least-privilege compute
Scoped IAM roles per Lambda
Full audit trail
CloudWatch + S3 for every retrieval
Model sees only what the user can see
Pre-filtered context passed to Bedrock



When to Use This Pattern

This architecture earns its complexity when:
  • Your corpus mixes sensitivity levels — HR, finance, legal, product strategy — in one searchable space.
  • You serve multiple roles or departments from a single RAG application.
  • You operate in a regulated industry (financial services, healthcare, government) with data-access compliance obligations.
  • You need to prove, not just claim, who accessed what and when.
If your RAG app serves one team over a uniformly public corpus, this much control may be overkill. But for anything enterprise-wide, the fence isn't optional — it's the architecture.



Conclusion

RAG is one of the most powerful patterns in enterprise AI — and one of the most dangerous when shipped without access control. AWS already gives you every piece needed to do this right: Cognito for identity, API Gateway for enforcement, OpenSearch for metadata-filtered retrieval, Bedrock for generation, CloudWatch for auditability.
Controlled-Access RAG doesn't limit how smart your AI can be. It limits how far it can reach. Build the fence first — then let the model roam freely inside it.


#buildatpce
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article