AWS Builder Center

Builder Center

Article 3: Building a RAG Application with Amazon Bedrock Knowledge Bases and Amazon S3

Article 3: Building a RAG Application with Amazon Bedrock Knowledge Bases and Amazon S3

In the first article, we designed an AI-powered University Knowledge Assistant. In the second, we built its serverless backend using AWS Lambda, API Gateway, and DynamoDB. Now, we'll implement the core AI capability: RAG using Amazon Bedrock Knowledge Bases and Amazon S3. This article walks through document storage, knowledge-base configuration, ingestion, retrieval, and generating context-aware answers. We'll also explore how to connect the RAG workflow to our existing backend.

Series: Generative AI on AWS (3 articles)

  1. 1
    Building an AI-Powered University Knowledge Assistant with AWS: From Idea to Architecture
  2. 2
    Building a Serverless Backend for an AI Application Using AWS Lambda, API Gateway, and DynamoDB
  3. 3
    Article 3: Building a RAG Application with Amazon Bedrock Knowledge Bases and Amazon S3 This article

1. Understanding the RAG Architecture

Before writing code, let's understand how the RAG pipeline works.
A traditional LLM generates answers based on its pretrained knowledge and the context provided in the prompt. A RAG application adds an information-retrieval step, allowing the model to use relevant content from external documents.
For our University Knowledge Assistant, the pipeline has two phases.

Phase 1: Document ingestion

  1. Upload university documents to Amazon S3.
  2. Connect the S3 bucket to an Amazon Bedrock Knowledge Base.
  3. Configure a text embedding model and vector storage.
  4. Ingest the documents, converting their content into chunks and vector embeddings.
  5. Index the embeddings so they can be retrieved using semantic search.

Phase 2: Question answering

  1. A student submits a natural-language question.
  2. The application sends the question to the knowledge base.
  3. The knowledge base retrieves relevant document chunks.
  4. A foundation model uses the retrieved content to generate a response.
  5. The application returns the answer and available source citations.
Amazon Bedrock Knowledge Bases provides APIs for retrieving relevant source chunks and for combining retrieval with response generation. The RetrieveAndGenerate operation handles the combined workflow and can return citations to source content. AWS documentation .

2. Choosing Our AWS Services

For this implementation, we'll use the following components.
ComponentAWS serviceResponsibility
Document storageAmazon S3Store academic documents
RAG orchestrationAmazon Bedrock Knowledge BasesManage ingestion and retrieval
Embedding modelAmazon BedrockConvert text into vector representations
Foundation modelAmazon BedrockGenerate natural-language answers
BackendAWS LambdaHandle questions and invoke the RAG workflow
APIAmazon API GatewayExpose the question-answering endpoint
Application dataAmazon DynamoDBStore conversation metadata
SecurityAWS IAMManage service permissions
MonitoringAmazon CloudWatchTrack application behavior and errors
For the initial version, we'll use a managed Knowledge Base connected to an S3 data source. The underlying vector storage and model configuration will be selected during setup, based on availability, requirements, and cost.
Amazon Bedrock also offers managed knowledge-base options with additional capabilities. Service availability, configuration choices, and supported models vary by Region. Check the current AWS documentation before deployment. AWS Knowledge Bases overview .

3. Preparing the University Documents

The quality of a RAG application depends heavily on the quality of its knowledge base.
For our prototype, we'll begin with a small collection of publicly available or authorized academic documents:
1
2
3
4
5
6
7
university-documents/
│
├── academic-regulations.pdf
├── examination-guidelines.pdf
├── semester-syllabus.pdf
├── attendance-policy.pdf
└── student-handbook.pdf
These documents should be official, current, and permitted for use in the application. Avoid uploading confidential student records or personal information.

Document preparation checklist

Before uploading, verify that:
  • PDFs contain extractable text or have been processed with OCR where necessary.
  • Documents are readable and not corrupted.
  • Duplicate or obsolete versions have been identified.
  • Each document has a clear title and, where possible, an effective date.
  • Documents are organized by category or academic year.
For a real deployment, maintaining document versions is particularly important. For example, an older examination policy should not be treated as current simply because it remains indexed.

4. Creating an Amazon S3 Bucket

Our first implementation step is to create an S3 bucket for the academic documents.
Open the AWS Management Console and navigate to Amazon S3.
Choose Create bucket and configure the following:
SettingExample
Bucket nameuniversity-assistant-documents-unique-id
AWS RegionYour selected Bedrock-supported Region
Object ownershipBucket owner enforced
Block public accessEnabled
VersioningEnabled
Default encryptionEnabled
S3 bucket names must be globally unique. Replace the example with an available name.
Keep Block Public Access enabled. Our application doesn't need publicly accessible documents; the Bedrock service role will receive the permissions required to ingest them.

Organizing the bucket

Create a prefix for the academic documents:
1
2
3
4
5
6
7
s3://university-assistant-documents-unique-id/
documents/
academic-regulations.pdf
examination-guidelines.pdf
semester-syllabus.pdf
attendance-policy.pdf
student-handbook.pdf
Using a dedicated prefix allows the knowledge base to ingest only the intended files rather than every object in the bucket.
Upload your documents through the S3 console or AWS CLI.
Example:
1
2
aws s3 cp academic-regulations.pdf \
s3://university-assistant-documents-unique-id/documents/
Repeat for the remaining documents.
For larger document collections, uploads can be automated using an application or a batch-processing workflow.

5. Creating an Amazon Bedrock Knowledge Base

With our documents stored in S3, we can create the knowledge base.

Step 1: Open Amazon Bedrock

Navigate to the Amazon Bedrock console in the AWS Region selected for the project.
Open the Knowledge Bases section and start creating a knowledge base.
The console workflow and available options may vary as AWS updates its managed knowledge-base experience.

Step 2: Configure the knowledge base

Provide a name such as:
1
UniversityKnowledgeBase
Add a description:
1
2
3
Knowledge base for university academic documents,
including regulations, syllabi, examination guidelines,
and student handbooks.
Select the available managed or custom knowledge-base configuration appropriate to your requirements.

Step 3: Configure the data source

Choose Amazon S3 as the data source.
Provide the S3 bucket and, where supported by the selected workflow, the documents/ inclusion prefix.
The S3 data source must be in the same Region as the knowledge base. The knowledge-base service role also needs appropriate permissions to access the bucket and any applicable encryption keys. AWS S3 data source guide .

Step 4: Select the embedding model

An embedding model converts text into numerical vectors that represent semantic meaning.
For example, these two questions have similar intent:
  • "What is the attendance requirement?"
  • "How much class attendance is necessary to qualify for examinations?"
A semantic retrieval system can identify related content even when the wording differs.
Choose an embedding model supported by the knowledge-base configuration and available in your Region. Consider supported languages, embedding dimensions, model cost, and retrieval quality.

Step 5: Configure vector storage

Select the vector-store option offered by the chosen knowledge-base workflow.
Depending on the configuration, options may include a managed vector store or supported AWS database integrations. For a prototype, a managed option can reduce operational overhead.
If using a custom vector store, verify its prerequisites, networking, IAM permissions, indexing configuration, and compatibility with the selected embedding model.

Step 6: Create the knowledge base

Review the configuration and create the resource.
AWS may create or configure the associated service role and supporting resources depending on the selected setup.
Once creation is complete, record the following identifiers:
1
2
3
KNOWLEDGE_BASE_ID
DATA_SOURCE_ID
AWS_REGION
We'll use the knowledge-base ID when integrating the application with the retrieval API.

6. Ingesting Documents into the Knowledge Base

Creating the knowledge base does not automatically mean all documents are ready for retrieval. We must run an ingestion or synchronization job.
In the Bedrock console:
  1. Open the newly created knowledge base.
  2. Locate the S3 data source.
  3. Start a synchronization or ingestion job.
  4. Wait for the job to complete.
  5. Review the job status and any ingestion warnings.
During ingestion, the service processes the source documents, extracts content, divides it into chunks, generates embeddings, and indexes the resulting representations.
<AsyncImage query="AWS cloud console inspired conceptual visualization document ingestion pipeline PDF files flowing into Amazon S3 bucket then AI document parsing text chunks embedding vectors and searchable vector database modern clean technical diagram dark navy background orange blue arrows no words no logos" aspectRatio="16:9" width="100%"/>
Figure 2: Conceptual document-ingestion pipeline.
Amazon Bedrock supports incremental synchronization for S3 data sources. When documents are added, updated, or deleted, a subsequent sync processes the changes rather than necessarily re-ingesting the entire collection. AWS ingestion and sync documentation .

Understanding document chunking

Chunking divides long documents into smaller passages that can be retrieved independently.
For example, an academic regulations PDF may contain several chapters:
1
2
3
4
5
6
7
8
9
Academic Regulations
|
+-- Chapter 1: Admission
|
+-- Chapter 2: Attendance
|
+-- Chapter 3: Examination
|
+-- Chapter 4: Evaluation
Instead of embedding the entire document as one representation, the ingestion process creates smaller chunks containing relevant passages.
Chunk size and overlap can affect retrieval quality:
  • Very large chunks may contain irrelevant information alongside the answer.
  • Very small chunks may lose important context.
  • Overlap can preserve continuity across chunk boundaries but may increase indexing and storage requirements.
For our first prototype, we'll use a supported default chunking strategy and evaluate retrieval quality before making custom adjustments.
For academic regulations, preserving section headings and document structure is particularly useful because the meaning of a policy often depends on its surrounding context.

7. Testing the Knowledge Base

Once ingestion completes successfully, we can test the knowledge base directly in the Amazon Bedrock console.
Start with a question that should have an answer in one of the uploaded documents:
1
2
What is the minimum attendance requirement
for appearing in semester examinations?
The knowledge base should retrieve the relevant passages and, when using response generation, produce an answer based on those passages.
Next, test a question with different wording:
1
2
How many classes must a student attend
to qualify for the semester exams?
This helps evaluate whether semantic retrieval works beyond exact keyword matching.
Finally, test a question that is not covered by the documents:
1
2
What is the university's official policy
for an entirely unrelated subject?
The application should avoid fabricating an answer when relevant evidence is unavailable.

What should we evaluate?

Evaluation criterionWhat to check
Retrieval relevanceAre the retrieved passages related to the question?
Answer correctnessDoes the response accurately reflect the source?
Source attributionCan users identify the supporting document?
Unsupported answersDoes the system acknowledge missing information?
Document freshnessDoes it retrieve the current version of the policy?
Response latencyHow long does retrieval and generation take?
Testing with a curated set of questions and verified answers will provide a more reliable assessment than trying only a few successful examples.

8. Integrating RAG into Our Python Backend

Now let's connect the knowledge base to the serverless backend we built in Article 2.
Our existing Lambda function accepts a question and returns a placeholder answer. We'll replace that placeholder with a call to Amazon Bedrock's RetrieveAndGenerate operation.
This API combines retrieval and generation and can return citations to the source documents. AWS retrieval documentation .

Step 1: Configure Lambda permissions

The Lambda execution role needs permission to invoke the knowledge-base retrieval and generation operation for the specific knowledge base.
Use a narrowly scoped IAM policy and verify the required actions and resource support against the current AWS service authorization reference.
The role should not have unrestricted access to all Bedrock resources.

Step 2: Configure environment variables

Add these environment variables to the Lambda function:
1
2
3
KNOWLEDGE_BASE_ID=your-knowledge-base-id
AWS_REGION=your-bedrock-region
MODEL_ARN=your-supported-model-arn
The model ARN should identify a foundation model supported for the selected knowledge-base generation workflow in the chosen Region.

Step 3: Implement the RAG function

The following example shows the core integration using Python and Boto3.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
import os
import boto3

bedrock_agent = boto3.client(
"bedrock-agent-runtime",
region_name=os.environ["AWS_REGION"]
)

KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
MODEL_ARN = os.environ["MODEL_ARN"]

def generate_answer(question, session_id=None):
request = {
"input": {
"text": question
},
"retrieveAndGenerateConfiguration": {
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": KNOWLEDGE_BASE_ID,
"modelArn": MODEL_ARN
}
}
}

if session_id:
request["sessionId"] = session_id

result = bedrock_agent.retrieve_and_generate(
**request
)

output = result.get("output", {}).get("text", "")

citations = []

for citation in result.get("citations", []):
for reference in citation.get("retrievedReferences", []):
location = reference.get("location", {})
s3_location = location.get("s3Location", {})

citations.append({
"text": reference.get("content", {}).get("text", ""),
"source": s3_location.get("uri", "")
})

return {
"answer": output,
"session_id": result.get("sessionId"),
"citations": citations
}
This function sends the question to the knowledge base, obtains the generated response, and extracts available source references.
The exact response structure and supported configuration options should be verified against the Boto3 version and API reference used in your deployment.

Step 4: Connect the function to the existing chat endpoint

In the Lambda handler from Article 2, replace the placeholder answer with the following call:
1
2
3
4
5
6
7
8
9
10
11
12
13
rag_result = generate_answer(
question,
session_id=body.get("session_id")
)

return response(200, {
"conversation_id": conversation_id,
"question": question,
"answer": rag_result["answer"],
"session_id": rag_result["session_id"],
"citations": rag_result["citations"],
"status": "success"
})
Here, response() is the existing JSON response helper from Article 2.
The client can retain the returned session ID and send it with subsequent requests to continue the same Bedrock session, subject to the service's session behavior and limits.
For a production system, persist the session identifier against an authenticated user and conversation in DynamoDB. Do not trust a client-provided session ID without checking that the authenticated user is authorized to access that conversation.

9. Understanding Conversation Context

A useful AI assistant should support follow-up questions.
For example:
Student: What is the minimum attendance requirement?
Assistant: According to the retrieved academic regulations, the required attendance is specified in the applicable policy.
Student: What happens if I don't meet it?
The second question depends on the context of the first.
Amazon Bedrock's RetrieveAndGenerate API supports session-based conversational context. The application can reuse the returned session ID for follow-up queries, allowing the service to maintain context across requests within a session.
However, session context should not replace application-level conversation storage. For a scalable application, use DynamoDB or another suitable database to maintain conversation metadata, user associations, retention policies, and audit information as needed.

10. Improving Document Retrieval

The first version of our knowledge base uses a straightforward ingestion and retrieval workflow. As the collection grows, several improvements may become useful.

Metadata filtering

Academic documents can be tagged with metadata such as:
1
2
3
4
5
6
{
"university": "Example University",
"academic_year": "2026-2027",
"document_type": "examination_policy",
"department": "information_technology"
}
Metadata can help narrow retrieval to the relevant academic year, department, or document category.
For an S3 data source, Bedrock Knowledge Bases supports document metadata files and retrieval filtering. Ensure that metadata values are validated and that filters are applied according to the user's authorization.

Reranking

Retrieval may initially return passages that are semantically similar but not the most useful.
A reranking model can reorder retrieved results based on their relevance to the query. This can improve the quality of the context supplied to the foundation model, though it adds configuration complexity and potentially additional cost and latency.

Better document parsing

Some university documents contain tables, scanned pages, diagrams, or complex layouts.
A simple text extraction pipeline may lose important information. For these documents, evaluate OCR, layout-aware parsing, or supported multimodal document-processing capabilities.

Handling outdated documents

Academic policies change over time. We should track publication dates, effective dates, and document versions. When a newer policy replaces an older one, the ingestion and retrieval process must avoid presenting superseded information as current.
Document metadata, carefully managed S3 objects, and regular synchronization can help support this process.

11. Security and Operational Considerations

Before making the assistant available to students, we need to consider the security and operational requirements.
Data access: All uploaded documents should be reviewed and authorized for use. S3 should remain private, and the Bedrock service role should have only the required access.
Document-level permissions: If different users are permitted to access different documents, authorization must be enforced during retrieval. A shared knowledge base without appropriate access filtering could expose content to unauthorized users.
Prompt injection: Documents may contain malicious or irrelevant instructions. Treat retrieved content as untrusted data, and instruct the model to use it as evidence rather than follow embedded commands. Test with adversarial documents and queries.
Sensitive information: Avoid ingesting confidential student records unless the application has a properly designed authorization, privacy, and retention model.
Observability: Monitor ingestion jobs, retrieval failures, model errors, latency, and application-level response quality. Avoid logging sensitive questions or document content unnecessarily.
Cost management: Embedding generation, model inference, vector storage, and retrieval all contribute to cost. Track actual usage and establish budgets before expanding the application.
Accuracy: RAG reduces dependence on a model's general knowledge for domain-specific answers, but it does not eliminate hallucinations or guarantee that retrieved sources are authoritative. Use verified source documents, evaluation datasets, and clear fallback behavior.

12. What We've Built

In this article, we've taken the backend from Article 2 and designed its core document-grounded AI workflow.
We've covered:
  • Creating a private S3 bucket for academic documents.
  • Connecting the S3 data source to Amazon Bedrock Knowledge Bases.
  • Configuring embeddings and vector storage.
  • Ingesting and synchronizing documents.
  • Testing semantic retrieval and generated answers.
  • Integrating RetrieveAndGenerate into a Python Lambda backend.
  • Returning source references and managing conversational sessions.
  • Exploring metadata filtering, reranking, security, and operational considerations.
The resulting prototype can accept a student question, retrieve relevant passages from a knowledge base, and generate a response grounded in the indexed documents.

Conclusion

Retrieval-Augmented Generation provides a practical approach to building AI applications that work with domain-specific information. With Amazon S3 and Amazon Bedrock Knowledge Bases, we can build a managed document ingestion and retrieval workflow without implementing every component of the RAG pipeline ourselves.
The most important lesson is that a reliable RAG application is more than an LLM connected to a document collection. It requires well-maintained source data, appropriate chunking and retrieval, careful access control, meaningful evaluation, and a clear strategy for handling missing or outdated information.
In the next article, we'll focus on connecting the RAG workflow to a custom application backend and frontend, improving the user experience with conversation management, source citations, and robust error handling.
Next: Connecting Amazon Bedrock to a Custom Backend: Building a Full-Stack AI Chat Application.
# agentic-ai# ai# artificial-intelligence# retreival-augmented-generation# ai-for-developers

Series: Generative AI on AWS (3 articles)

  1. 1
    Building an AI-Powered University Knowledge Assistant with AWS: From Idea to Architecture
  2. 2
    Building a Serverless Backend for an AI Application Using AWS Lambda, API Gateway, and DynamoDB
  3. 3
    Article 3: Building a RAG Application with Amazon Bedrock Knowledge Bases and Amazon S3 This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article