AWS Builder Center

Builder Center

Build a RAG Chatbot on AWS with Amazon Bedrock Knowledge Bases

Build a RAG Chatbot on AWS with Amazon Bedrock Knowledge Bases

A beginner-friendly, step-by-step guide to building a RAG chatbot that answers questions from your own documents using Amazon Bedrock Knowledge Bases, S3 and Python, including testing, optional deployment and cleanup to avoid surprise costs.

Tharsan

Introduction

Large language models know a lot, but they don't know your data: your notes, company documents or project files. Ask one about a private PDF and it may say it doesn't know, or make something up.
The fix is RAG (Retrieval-Augmented Generation). Instead of retraining a model, you:
  1. Search your documents for the parts relevant to the question
  2. Give those parts to the model along with the question
  3. Let the model answer using that context
Amazon Bedrock Knowledge Bases handles most of this for you. In this post we'll build a chatbot that answers questions from your own documents.

How it works

  • S3 stores your documents.
  • An embedding model converts text chunks into vectors.
  • A vector store holds the vectors for similarity search.
  • A foundation model writes the final answer from the retrieved chunks.
Flow: Documents (S3) → chunk and embed → vector store → retrieve relevant chunks → foundation model → answer with sources.

Prerequisites

  • An AWS account with billing alerts set up
  • An IAM user or role with Bedrock permissions (not the root account)
  • A region where Bedrock and Knowledge Bases are available
  • A few small test documents (PDF, TXT, DOCX or MD)

Step 1: Enable model access

  1. Open the Amazon Bedrock console.
  2. Go to Model access.
  3. Enable an embedding model and a text generation model.
Do this first, because some models may need a short approval.

Step 2: Upload your documents to S3

  1. Create an S3 bucket, for example my-rag-demo-docs-yourname.
  2. Upload your documents.
  3. Keep the bucket private. Knowledge Bases reads it through an IAM role.
Clean, well-structured documents give much better answers than messy scans.

Step 3: Create the Knowledge Base

  1. Go to Knowledge Bases → Create knowledge base.
  2. Name it and let Bedrock create a new IAM service role.
  3. Choose S3 as the data source and select your bucket.
  4. Choose your embedding model.
  5. Choose a vector store. The quick-create option is easiest for a first project.
  6. Review and create.
Cost warning: Some vector store options bill continuously while they exist, even when idle. Check the pricing before you choose.

Step 4: Sync your data

Click Sync on the data source. Bedrock reads your files, splits them into chunks, creates embeddings and stores them. Run Sync again whenever your documents change.

Step 5: Test in the console

  1. Open the Test panel.
  2. Select a generation model.
  3. Ask a question about your documents.
  4. Check the answer and the source citations.

Step 6: Call it from Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3

client = boto3.client("bedrock-agent-runtime", region_name="us-east-1")

response = client.retrieve_and_generate(
input={"text": "What is the refund policy in the document?"},
retrieveAndGenerateConfiguration={
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": "YOUR_KB_ID",
"modelArn": "YOUR_MODEL_ARN",
},
},
)

print(response["output"]["text"])
Replace YOUR_KB_ID and YOUR_MODEL_ARN with your own values, and set the region you are using. Use the retrieve API instead if you want only the matching chunks so you can build your own prompt.

Step 7 (optional): Make it a real chatbot

  • A Lambda function that runs the code above
  • API Gateway to expose it as an HTTP endpoint
  • A simple chat page hosted on S3 + CloudFront
Pass the sessionId from the response on later requests to keep conversation context.

Tips for better answers

  • Use clear, well-organised source documents
  • Tune chunking settings if answers feel cut off or irrelevant
  • Instruct the model to answer only from the provided context and to say when it doesn't know
  • Always check the citations. RAG reduces made-up answers but doesn't eliminate them.

Cleanup (don't skip this)

Delete the following to avoid charges:
  1. The knowledge base
  2. The vector store it created
  3. The S3 bucket and its contents
  4. The IAM role, if you no longer need it
Check Billing → Cost Explorer a day later to confirm nothing is still running.

Key takeaways

  • RAG lets a model answer from your data without retraining
  • Knowledge Bases removes most of the plumbing: chunking, embeddings and retrieval
  • Document quality matters a lot
  • Set a budget before you start

What's next

  • Add a web front end
  • Compare different chunking strategies
  • Add guardrails to control what the chatbot can say
# agentic-ai# agents# agent-to-agent# ai# ai-assisted
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article