AWS Builder Center
Your AI Does Not Know Your Documents

Your AI Does Not Know Your Documents

How RAG with Amazon Bedrock turns private knowledge into trusted answers

Software Engineer | Cloud & AI | AWS Certified | Community Builder | Top 1% on Topmate | AWS Captain
Imagine that you have built a very intelligent assistant.
It can explain difficult ideas. It can write code. It can answer questions in seconds. It sounds confident and helpful.
Then someone asks a simple question.
“What does our deployment guide say about the production environment?”
The assistant gives a beautiful answer. Unfortunately, the answer is wrong.
This is the moment when many teams discover an important truth about artificial intelligence. A model can be very intelligent and still know nothing about your documents, your policies, your product, or your internal knowledge.
This article is about solving that problem.
We will build a simple Retrieval Augmented Generation application, usually called RAG. Our application will store documents in Amazon S3, use Amazon Bedrock Knowledge Bases to find relevant information, use S3 Vectors as a simple vector storage option, and ask a foundation model to generate an answer based on the retrieved content.
We will also call the same knowledge base from Python and Node.js with TypeScript. The purpose is not to build another impressive demo that works only once. The purpose is to understand how a useful RAG system is designed, where it can fail, and how we can make it more trustworthy.

The real problem is not intelligence

A foundation model has broad knowledge learned during training. That does not mean it knows the latest version of your documentation. It does not automatically know what changed in your database yesterday. It does not know the private documents that your organization created last week.
One possible solution is to place all of your documents inside the prompt. This works for a small experiment. It becomes difficult when the collection grows. Prompts become large. Costs increase. Responses become slower. It also becomes difficult to answer a simple question such as, “Which document supports this answer?”
RAG gives us a better way to think about the problem.
Instead of asking the model to remember everything, we allow it to search a trusted collection first. The most relevant parts of that collection are then added to the model request. The model uses that context to create a response.
AWS explains the RAG process as a sequence in which documents are converted into text, divided into smaller chunks, converted into embeddings, stored in a vector index, and retrieved later when a user asks a question .
The idea is simple.
Do not ask the model to guess what your documents say. Give it the right part of the document and ask it to explain that information.

What we are going to build

Our example will be a technical documentation assistant. We will give it a small collection of documents, such as an architecture overview, a deployment guide, and a troubleshooting guide. A user will ask a question, and the application will return an answer together with references to the source documents.
The complete journey has two parts.
Part
What happens
Document preparation
We place useful documents in Amazon S3 and synchronize them with the knowledge base.
Question answering
The application sends a question to Amazon Bedrock, which retrieves relevant content and generates a cited answer.
Amazon Bedrock Knowledge Bases manages much of the RAG workflow for us. It can process documents, create embeddings, store them in a vector index, retrieve relevant chunks, and use those chunks to support a generated answer .
For this tutorial, we will use S3 Vectors because it is a relatively simple vector storage option for a learning project. AWS now separates Managed Knowledge Bases from customer managed Knowledge Bases. In this article, we are showing the customer managed vector store path so that the storage decision remains visible. For a new production system, compare this approach with the Managed Knowledge Base option that AWS currently recommends for an optimized and more managed experience .

The architecture in plain language

The architecture can be understood without memorizing service names.
First, our documents live in a private S3 bucket. Bedrock reads those documents and prepares them for search. It splits the text into smaller pieces and creates a numerical representation for each piece. These numerical representations are called embeddings. Similar ideas produce vectors that are close to each other in the vector index.
When a user asks a question, the question is also converted into a vector. The system searches for document chunks that are semantically similar to the question. Those chunks become the evidence for the final answer.
The flow looks like this:
Documents in S3
|
v
Bedrock Knowledge Base
|
v
Chunks, embeddings and S3 Vectors index
|
v
User question
|
v
RetrieveAndGenerate request
|
v
Answer with source references
The application can run locally while we learn. Later, the same query function can be placed behind API Gateway and Lambda. The browser or mobile application should call our API. It should not receive AWS credentials or direct access to the knowledge base.
Before we write code, choose good documents
Many RAG problems are blamed on the model even though the real problem is the document collection.
If the documents are unclear, badly formatted, outdated, or full of unrelated information, retrieval will be difficult. A strong model cannot retrieve evidence that does not exist in a useful form.
Start with a small collection. Three to ten focused documents are enough for the first version.
docs/
architecture overview.md
deployment guide.md
troubleshooting.md
Each document should have clear headings. Each important idea should have enough surrounding explanation to make sense on its own. Avoid placing unrelated subjects in one large file. If your documents contain versions, dates, or product names, write them clearly.
For a public tutorial, use your own technical content, public documentation that you are allowed to use, or fictional documents. Never upload private company data to a personal learning account without permission.
Create a private S3 bucket and upload the documents.
Bash
aws s3 cp ./docs/ s3://YOUR_BUCKET_NAME/docs/ --recursive --region "$AWS_REGION"
The command is simple. The security around the bucket is important. Keep the bucket private. Enable encryption. Give the Bedrock service role access only to the required location.

Create the knowledge base

Open the Amazon Bedrock console and choose Knowledge Bases. Create a new knowledge base and connect it to the S3 location that contains your documents.
When the console asks you to choose a vector store, select S3 Vectors if the option is available in your Region and account. You can use the quick creation flow for this learning project.
The exact console labels may change over time, but the idea remains the same.
  1. Open Amazon Bedrock.
  1. Choose Knowledge Bases.
  1. Create a new knowledge base.
  1. Connect the knowledge base to your S3 document location.
  1. Select an embedding model available in your Region.
  1. Select S3 Vectors as the vector store.
  1. Allow Bedrock to create the service role, or provide a carefully scoped role.
  1. Start the ingestion or synchronization job.
Wait for the ingestion job to finish successfully. A knowledge base can be created correctly and still return no useful answer if the ingestion job never completed.
S3 Vectors provides a managed vector storage path for suitable RAG workloads. Bedrock can read the S3 source, create text blocks, generate embeddings, and store those embeddings in the vector index . It is useful for learning because we do not need to operate a separate vector database cluster.
There is an important tradeoff. S3 Vectors supports semantic search, but it does not provide hybrid search. It also has documented metadata limitations. These constraints may be perfectly acceptable for a small documentation assistant, but they should be reviewed before using the same architecture for a large enterprise search platform .

The first important lesson: test retrieval before generation

Suppose the assistant gives a wrong answer. What should we check first?
Many developers immediately change the model or rewrite the prompt. That may not solve the problem. The correct document chunk may never have reached the model.
Test retrieval first.
Ask a question whose answer is clearly present in your documents.
Plain Text
Which deployment environment uses the blue green strategy?
Use the Bedrock console to inspect the retrieved chunks. Did the system return the paragraph that contains the answer? If it did not, the problem is probably related to the document, the chunking strategy, the query, or metadata.
This simple habit saves a lot of time.
Before improving generation, confirm that retrieval found the right evidence.
Large chunks may contain too much unrelated information. Very small chunks may remove the context that gives a sentence its meaning. For a first implementation, begin with the default chunking approach, create a small evaluation set, and change one setting at a time.

Query the knowledge base with Python

Install Boto3, the AWS SDK for Python.
Bash
python -m pip install boto3
Set the required environment variables.
Bash
export AWS_REGION="us-east-1"
export KNOWLEDGE_BASE_ID="replace-with-your-knowledge-base-id"
export BEDROCK_MODEL_ARN="replace-with-a-supported-model-id-or-arn"
Create a file named rag_query.py.
import os
from typing import Any
import boto3
AWS_REGION = os.environ.get("AWS_REGION", "us east 1")
KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
MODEL_ARN = os.environ["BEDROCK_MODEL_ARN"]
client = boto3.client(
"bedrock-agent-runtime",
region_name=AWS_REGION,
)
def ask_knowledge_base(
question: str,
session_id: str | None = None,
) -> dict[str, Any]:
request: dict[str, Any] = {
"input": {"text": question},
"retrieveAndGenerateConfiguration": {
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": KNOWLEDGE_BASE_ID,
"modelArn": MODEL_ARN,
"retrievalConfiguration": {
"vectorSearchConfiguration": {
"numberOfResults": 5
}
},
},
},
}
if session_id:
request["sessionId"] = session_id
return client.retrieve_and_generate(**request)
def print_response(response: dict[str, Any]) -> None:
print("Answer:\n")
print(response["output"]["text"])
print("\nSources:\n")
for number, citation in enumerate(
response.get("citations", []),
start=1,
):
for reference in citation.get("retrievedReferences", []):
location = reference.get("location", {})
s3_location = location.get("s3Location", {})
source_uri = s3_location.get("uri", "unknown source")
source_text = reference.get("content", {}).get("text", "")
short_text = source_text[:240].replace("\n", " ")
print(f"{number}. {source_uri}")
print(f" {short_text}")
if __name__ == "__main__":
question = input("Ask a question: ").strip()
response = ask_knowledge_base(question)
print_response(response)
print("\nSession ID:", response.get("sessionId"))
The method that does the work is retrieve_and_generate. It receives a question, identifies the knowledge base, retrieves relevant chunks, and asks the selected model to generate a response. AWS documents that the response can include generated text, citations, retrieved references, and a session ID.
The citations matter. A response without evidence may sound convincing, but a response with references can be inspected and challenged.
If you are building a multi turn conversation, save the session ID returned by Bedrock and pass it into the next request. The session ID allows the service to maintain context for that conversation. Do not create your own session ID.

Query the same knowledge base with TypeScript

The architecture does not change just because the programming language changes. That is a useful idea for teams. The same knowledge base can serve applications written in different languages.
Create a TypeScript project.
Bash
mkdir rag-ts-client
cd rag-ts-client
npm init -y
npm install @aws-sdk/client-bedrock-agent-runtime
npm install --save-dev typescript tsx @types/node
Create src/rag-query.ts.
import {
BedrockAgentRuntimeClient,
RetrieveAndGenerateCommand,
RetrieveAndGenerateCommandInput,
} from "@aws-sdk/client-bedrock-agent-runtime";
const region = process.env.AWS_REGION ?? "us-east-1";
const knowledgeBaseId = process.env.KNOWLEDGE_BASE_ID;
const modelArn = process.env.BEDROCK_MODEL_ARN;
if (!knowledgeBaseId || !modelArn) {
throw new Error(
"KNOWLEDGE_BASE_ID and BEDROCK_MODEL_ARN must be configured"
);
}
const client = new BedrockAgentRuntimeClient({ region });
async function askKnowledgeBase(
question: string,
sessionId?: string
) {
const input: RetrieveAndGenerateCommandInput = {
input: { text: question },
retrieveAndGenerateConfiguration: {
type: "KNOWLEDGE_BASE",
knowledgeBaseConfiguration: {
knowledgeBaseId,
modelArn,
retrievalConfiguration: {
vectorSearchConfiguration: {
numberOfResults: 5,
},
},
},
},
...(sessionId ? { sessionId } : {}),
};
return client.send(new RetrieveAndGenerateCommand(input));
}
function printResponse(response: Awaited<ReturnType<typeof askKnowledgeBase>>) {
console.log("Answer:\n");
console.log(response.output?.text ?? "No answer returned");
console.log("\nSources:\n");
for (const [index, citation] of (response.citations ?? []).entries()) {
for (const reference of citation.retrievedReferences ?? []) {
const sourceUri = reference.location?.s3Location?.uri ?? "unknown source";
const sourceText = reference.content?.text ?? "";
console.log(`${index + 1}. ${sourceUri}`);
console.log(` ${sourceText.slice(0, 240).replace(/\n/g, " ")}`);
}
}
}
const question = process.argv.slice(2).join(" ").trim();
if (!question) {
throw new Error('Usage: npx tsx src/rag-query.ts "your question"');
}
const response = await askKnowledgeBase(question);
printResponse(response);
console.log("\nSession ID:", response.sessionId ?? "not returned");
Run it with your configured environment variables.
Bash
npx tsx src/rag-query.ts "Which deployment environment uses the blue green strategy?"
The TypeScript example gives us the same result as the Python example. The important concepts remain the same: retrieve evidence, generate an answer, preserve citations, and manage the session when conversation history is required.

Turn the client into a serverless API

A local script is useful for learning. A real product usually needs an API.
The next step is to place the query function inside AWS Lambda and expose it through Amazon API Gateway. The browser sends a question to the API. API Gateway passes it to Lambda. Lambda calls Bedrock. The response returns to the browser.
Browser or mobile application
|
v
API Gateway
|
v
Lambda
|
v
Bedrock RetrieveAndGenerate
|
v
Answer and citations
The browser should not know your AWS credentials. The browser should not receive unrestricted access to your S3 bucket. Lambda should validate the request, enforce authentication, apply reasonable input limits, and return only the source information that the user is allowed to see.
A clean response shape might look like this:
{
"answer": "The production environment uses blue green deployment.",
"sources": [
{
"uri": "s3://example bucket/docs/deployment guide.md",
"snippet": "The production environment uses blue green deployment..."
}
],
"sessionId": "returned by bedrock"
}
In a real application, you may want to transform or hide the S3 URI. A source reference is useful, but it should not reveal private bucket structure or sensitive document names.

The prompt should know its limits

A RAG application should be honest when the documents do not contain an answer.
A useful instruction is:
Plain Text
Answer the question using only the retrieved context. If the context does not contain enough information, say that the answer is not available in the connected documents. Do not invent commands, product names, configuration values, or citations.
This instruction improves behavior, but it is not a complete security control. Documents should be treated as data, not as trusted instructions. A document could contain text that attempts to manipulate the model. Test prompt injection scenarios and keep application rules outside the document collection.
AWS points out that guardrails apply to the user input and generated response, but not to references retrieved from Knowledge Bases at runtime. This means we should not assume that every retrieved passage is safe to display. Add access control, redaction, output validation, and document governance according to the use case.

Security is part of the design

A RAG system can expose more than an incorrect answer. It can also expose a document to the wrong user if permissions are designed poorly.
Area
Practical approach
Credentials
Use IAM roles or profiles. Never place credentials in source code.
S3
Keep documents private, use encryption, and limit access to the required prefix.
IAM
Give the caller and the Bedrock service role only the permissions they need.
API
Put authentication and authorization in front of the Lambda function.
Sensitive data
Do not use private or regulated data in a learning project without approval.
Logging
Avoid logging complete prompts and retrieved passages when they contain sensitive information.
Sources
Return only source information that the current user is allowed to see.
Abuse
Add request limits, monitoring, budget alerts, and throttling.
There are usually two identities to understand. The application identity calls the Bedrock runtime API. The Bedrock service role accesses the S3 data source and vector store. A permission error may come from either one.

How do we know that the application works?

A fluent answer is not automatically a correct answer.
Create a small evaluation set. Ten to twenty questions are enough for a first project. Include questions with different behavior.
Test
What it teaches us
Direct fact
Whether the system can find an exact answer.
Paraphrased question
Whether retrieval understands meaning rather than exact words.
Multi document question
Whether the system can combine evidence carefully.
Unknown question
Whether the system admits that information is unavailable.
Conflicting information
Whether versions and source priority are handled clearly.
Prompt injection test
Whether document text can improperly control the model.
Long question
Whether the system handles realistic user input.
For every test, record four things. Did the correct source appear? Was the final answer supported by that source? Did the system avoid inventing information? How long did the request take?
Keep a simple file such as this:
Plain Text
question,expected source,actual source,answer supported,latency,notes
When you tune the system, change one thing at a time. Do not change chunk size, number of results, prompt wording, and model settings in one experiment. A small record of your experiments is valuable both for your own work and for the community reading your article.

Cost and performance

A RAG application has more than one cost. You may pay for document storage, embedding generation, vector storage, model input, model output, Lambda requests, API Gateway requests, logging, and data transfer.
For a public article, avoid presenting an exact cost without stating your assumptions. Mention the Region, model, document count, request volume, and retention period used in your example. Check current AWS pricing before publication because pricing and service availability can change.
You can often improve efficiency by reducing unnecessary ingestion, keeping prompts focused, limiting the number of retrieved results, caching safe repeated responses, and selecting a model that is appropriate for the task.
AWS documentation also suggests reducing the number of retrieved results, using smaller chunks, shortening the prompt template, or shortening the user question when the prompt becomes too large.
S3 Vectors may be attractive when storage cost and simple managed operations matter more than advanced search features. It is not a universal solution. The documented semantic search behavior and metadata limits should be compared with the requirements of the target workload.

Common problems and what they teach us

The answer says that the information is unavailable
Check whether ingestion completed. Then inspect the retrieved chunks directly. If the right paragraph is missing, review the document structure, chunking approach, metadata, and wording of the question.
The request is denied
Check the permissions of the application caller. Then check the Bedrock service role. Confirm access to the S3 location, vector store, embedding model, and generation model.
The response is slow
Separate retrieval time from generation time if possible. Reduce the number of results, keep the prompt focused, review Lambda timeout settings, and consider an asynchronous design for long running work.
The answer invents details
Inspect the retrieved references. If the correct evidence is missing, improve retrieval. If the evidence is present but the model still invents details, strengthen the instructions, add evaluation cases, and make unsupported answers explicit.
A follow up question loses context
Pass the session ID returned by the previous request. In a multi user system, store and protect session information carefully.

What makes this a real engineering project?

The interesting part of this project is not the first successful answer. The interesting part is understanding the boundaries.
We learned that document quality matters. We learned that retrieval must be tested before generation. We learned that citations are not decoration. We learned that the browser should not receive AWS credentials. We learned that a model can still produce an unsupported answer even when a prompt says, “Do not hallucinate.”
These lessons are useful because they apply to almost every AI application that works with private or changing information.
A strong community article should therefore include more than a screenshot of a chatbot. Include the document examples. Include the architecture. Include the code. Explain the permissions. Show a failed retrieval example. Explain how you measured improvement. Share the cleanup steps. Publish a sanitized GitHub repository that another builder can run.
That is how a tutorial becomes a contribution.

Final thoughts

RAG is often explained as a technical pattern, but its deeper value is about trust.
When a user asks a question, the system should not simply sound confident. It should use the right evidence. It should be able to show where the answer came from. It should admit when the available documents do not contain enough information.
With Amazon S3, Amazon Bedrock Knowledge Bases, S3 Vectors, Python, and TypeScript, we can build a simple starting point without operating a complicated search platform. From there, we can add API Gateway, Lambda, authentication, monitoring, evaluation, and better document governance.
The most important lesson is this:
A reliable AI application is not the one that always gives an answer. It is the one that knows when to answer, knows what evidence to use, and knows when to say that it does not know.

References

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article