AWS Builder Center

Builder Center

Retrieve Data from Documents with Amazon Bedrock and Knowledge Bases

Discover how Amazon Bedrock Knowledge Bases enable scalable document ingestion, semantic retrieval, and RAG-based answer generation across your organization’s content sources.

Introduction

Many organisations have vast volumes of unstructured data in their database about their company, policies, operational handbooks, manuals, reports, and archived data. Employees and customers expect instant and accurate replies to their doubts and questions such as:
  • “What is our updated refund policy?”
  • “How do I request access renewal?”
  • “Where can I find the security compliance checklist?"
Traditional systems used in some organisations might struggle to understand context and reply on time. Large Language Models might also hallucinate when answers are not grounded in real data.
Amazon Bedrock for RAG (Retrieval-Augmented Generation) is a fully managed AWS service that simplifies the process of building generative AI applications that use proprietary data sources. This ensures answers are always derived from the organization’s trusted information sources.

Problem Statement

Enterprises face several challenges when retrieving information from distributed documents:
  1. Information is distributed across multiple systems like S3 buckets, Confluence, Salesforce, Google Drive, and SharePoint.
  2. Limited Semantic Understanding of traditional systems impact the search results.
  3. LLMs alone cannot access enterprise data as without grounding, answers may be missing.
  4. There is no built-in evaluation metric for measuring answer accuracy and quality.

Solution

Amazon Bedrock Knowledge Bases simplify the entire RAG workflow by:
  • Automatically ingesting documents from S3, Confluence, SharePoint, Salesforce, and web sources
  • Chunking content into meaningful sections
  • Storing vectors in Amazon OpenSearch Serverless
  • Retrieving relevant chunks based on semantic similarity
  • Generating grounded answers through Bedrock foundation models
  • Supporting real-time streaming responses
  • Enabling evaluation using RAGAS metrics
This provides a complete retrieval and generation pipeline without building custom infrastructure.

How the Solution Works

The RAG workflow consists of six primary steps:

1. User sends a query

Example: “What is our refund policy for digital products?”

2. Bedrock determines whether document retrieval is needed

If external context is required, retrieval is triggered.

3. Knowledge Base retrieves relevant chunks

This involves:
  • Ingestion — KB reads files from connected data sources
  • Chunking — Content is split into 200–400 token sections
  • Embedding — Titan Embeddings generate vector representations
  • Vector Search — Compare query vector against stored vectors

4. Bedrock builds an augmented prompt

Includes:
  • The user's question
  • The retrieved document snippets

5. Foundation model generates a grounded answer

Models supported include:

  • Claude
  • Llama
  • Titan
  • AI21

6. Final answer is returned to the user

Can include citations to documents for transparency.

Architecture

  1. Documents: They refer to the raw source files like pdfs, policy docs, manuals, etc.
  2. Document Chunks: The large documents are broken down in simpler, smaller meaningful version for easier chunking and analysis.
  3. Document Embeddings: Each chunk is converted into numerical vectors or embeddings representing contextual and semantic representation of the document.
  4. Vector Database: It stores the chunk embeddings, the chunk ID and metadata.
  5. Question: This is the query provided to the system.
  6. Question Embedding: The user's query is transformed into embedding vectors which help in contextual comparison to the document chink embeddings.
  7. LLM (Large Language Model): The model takes question, retrieved chunks, prompt template and then generates a grounded answer using the provided context.

TECHNICAL IMPLEMENTATION

Building the Knowledge Base

  1. Configure the Vector Index
  2. Add data sources like S3, Confluence, Salesforce, SharePoint, etc
  3. Start the ingestion pipeline where Bedrock automatically reads documents, performs chunking, and stores the embeddings of the chunks

Chunking

Chunking in Amazon Bedrock RAG is the crucial process of splitting large documents into smaller, meaningful text segments (chunks) for efficient retrieval, using methods like Fixed-Size, Semantic, Heirarchical, and Custom strategies.
Recommended chunk sizes:
  1. Text: 200–400 tokens
  2. PDFs: 300–500 tokens
  3. Code: 40–80 lines
  4. Images: 1 image → 1 chunk

Retrieve API (Retrieval Only)

Amazon Bedrock provides the Retrieve API for a retrieve-only workflow within a Retrieval-Augmented Generation (RAG) system. This API queries a knowledge base and returns relevant source document chunks and their relevance scores, without generating a final response using a large language model (LLM).

1
2
3
4
5
6
7
8
9
10
11
12
13
def retrieve(query, kbId, numberOfResults=5):
return bedrock_agent_client.retrieve(
retrievalQuery={
'text': query
},
knowledgeBaseId=kbId,
retrievalConfiguration={
'vectorSearchConfiguration': {
'numberOfResults': numberOfResults,
'overrideSearchType': "HYBRID", # optional
}
}
)

Retrieve-and-Generate API (Full RAG)

It is an Amazon Bedrock API that searches your knowledge base and generates an answer using an LLM in one single step.It handles retrieval, filtering, grounding, and generation automatically, giving you a fully managed RAG pipeline.
1
2
3
4
5
response = client.retrieve_and_generate(
knowledgeBaseId="kb123",
input={"text": "Explain the refund policy"},
retrieveAndGenerateConfiguration={"type": "KNOWLEDGE_BASE"}
)

Query Reformulation

Instead of directly using the user query as the reference in semantic search, the agent formulates itself a reference sentence that can be closer to the targeted documents. LLM reformulates the query to retrieve results.
1
2
3
4
5
reformulated = bedrock_agent_client.rewrite_query(text=query)
response = bedrock_agent_client.retrieve(
retrievalQuery={'text': reformulated},
knowledgeBaseId=kbId
)

Reranking

1
2
3
results = bedrock_agent_client.retrieve(retrievalQuery={'text': query}, knowledgeBaseId=kbId)
reranked = bedrock_agent_client.rerank(query=query, results=results['retrievedResults'])
final_results = reranked['results']

Evaluating RAG with RAGAS

RAGAS (RAG Assessment) is an open-source library used to measure the quality of your RAG pipeline, including:
  • how well chunks are retrieved
  • how accurate the generated answers are
  • how grounded the responses are in the retrieved context
  • how faithful the LLM is (no hallucination)
You give RAGAS:
  • your query
  • retrieved context chunks
  • the generated answer
  • (optional) the ground truth answer
RAGAS evaluates everything using embeddings + LLM scoring.
1
2
3
4
5
6
result = evaluate(
dataset=dataset,
metrics=[faithfulness, answer_relevancy],
llm=llm,
embeddings=bedrock_embeddings
)

GuardRails for Amazon Bedrock

Scaling Strategy

Current Capacity

  • Thousands of documents
  • Tens of millions of embeddings
  • Concurrent retrieval workloads
  • Multi-modal dataset ingestion

Future Expansion

  • Multi-region Knowledge Bases
  • Automated ingestion
  • Sharded vector indexes
  • Graph RAG expansion for interlinked policies
  • Batch RAG for analytical queries

Visual Documentation

Creating Knowledge Base

1
2
3
4
5
6
7
knowledge_base = BedrockKnowledgeBase(
kb_name = f'{knowledge_base_name}',
kb_description = knowledge_base_description,
data_sources = data_sources,
chunking_strategy = "FIXED_SIZE",
suffix = f'{suffix}-f'
)

Upload Data to S3 Bucket

1
2
3
4
5
6
7
8
def upload_directory(path, bucket_name):
for root, dirs, files in os.walk(path):
for file in files:
file_to_upload = os.path.join(root, file)
print(f"uploading file {file_to_upload} to {bucket_name}")
s3_client.upload_file(file_to_upload, bucket_name, file)

upload_directory("../synthetic_dataset", data_bucket_name)

Amazon Bedrock's Retrieve-And-Generate API

1
2
3
4
5
6
7
# Create bedrock agent client
bedrock_config = Config(connect_timeout=120, read_timeout=120, retries={'max_attempts': 0}, region_name=region_name)
bedrock_agent_client = boto3_session.client("bedrock-agent-runtime", config=bedrock_config)

# Define FM to be used for generations
model_id = "anthropic.claude-3-haiku-20240307-v1:0" # we will be using Anthropic Claude 3 Haiku throughout the notebook
model_arn = f'arn:aws:bedrock:{region_name}::foundation-model/{model_id}'

Conclusion

Amazon Bedrock Knowledge Bases enable teams to build a complete RAG solution without managing ingestion pipelines, vector stores, retrieval logic, or prompt orchestration.
By integrating chunking, reranking, query reformulation, and RAGAS evaluation, organizations can develop reliable and scalable knowledge experiences that deliver accurate and grounded answers for enterprise use cases.
This architecture provides a strong foundation for AI-driven knowledge discovery across any industry.
# amazon-bedrock# knowledge-bases-for-bedrock# rag# vector-database# api
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article