Stop Hallucinating: Adding RAG to Amazon Bedrock with Knowledge Bases
Tired of generic AI hallucinating answers? Here is how I set up a context-aware chatbot using Amazon Bedrock Knowledge Bases to query my own documents—no manual vector database setup required.
Building a custom AI chatbot usually requires setting up document chunkers, embedding pipelines, and hosting vector databases. Amazon Bedrock Knowledge Bases automates all of that, handling retrieval-augmented generation (RAG) natively on AWS.
Here is how to set it up in 5 simple steps.
The Stack
The Stack
Amazon S3: Raw document storage (.pdf, .md, .txt).
Amazon Bedrock Knowledge Bases: Managed RAG orchestration.
Amazon Titan Text Embeddings v2: Converts text into vector embeddings.
Amazon OpenSearch Serverless: Auto-provisioned vector store.
Anthropic Claude 3.5 Sonnet: Foundation model for generating answers.
Amazon Bedrock Knowledge Bases: Managed RAG orchestration.
Amazon Titan Text Embeddings v2: Converts text into vector embeddings.
Amazon OpenSearch Serverless: Auto-provisioned vector store.
Anthropic Claude 3.5 Sonnet: Foundation model for generating answers.
Step-by-Step Setup
Enable Model Access: Go to Bedrock Console > Model Access and enable Anthropic Claude and Amazon Titan Embeddings.
Upload Documents: Create an S3 bucket (keep public access blocked) and upload your target PDFs or Markdown files.
Create the Knowledge Base: Navigate to Knowledge Bases > Create, select your S3 bucket as the data source, choose Titan Text Embeddings v2, and select Quick create a new vector store.
Sync Your Data: Open your new Knowledge Base, find your data source, and click Sync to chunk, embed, and index your files into OpenSearch.
Test: Use the Test Knowledge Base panel on the right side of the console. Select Claude 3.5 Sonnet, ask a question about your files, and view the answer along with source citations.
Upload Documents: Create an S3 bucket (keep public access blocked) and upload your target PDFs or Markdown files.
Create the Knowledge Base: Navigate to Knowledge Bases > Create, select your S3 bucket as the data source, choose Titan Text Embeddings v2, and select Quick create a new vector store.
Sync Your Data: Open your new Knowledge Base, find your data source, and click Sync to chunk, embed, and index your files into OpenSearch.
Test: Use the Test Knowledge Base panel on the right side of the console. Select Claude 3.5 Sonnet, ask a question about your files, and view the answer along with source citations.
Quick Takeaways
Provisioning Time: OpenSearch Serverless takes 5–10 minutes to spin up during initial creation.
Chunking: The default 300-token chunk size works for standard text, but dense tables or code blocks may need custom chunk settings.
Cost Tip: Clean up your OpenSearch Serverless collection after testing to avoid charges for idle compute units (OCUs).
Chunking: The default 300-token chunk size works for standard text, but dense tables or code blocks may need custom chunk settings.
Cost Tip: Clean up your OpenSearch Serverless collection after testing to avoid charges for idle compute units (OCUs).
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article