Exploring Data Retrieval and Knowledge Bases with Amazon Bedrock
Understanding knowledge base, RAG (Retrieval-Augmented Generation), evaluation of RAGs using RAGAS, Chunking in RAG, Autogenerated Filters, Metadata Filtering, Query Reformulation, Re-ranking, Guardrails contextual grounding, Data Prep: Audio/Video/Image/Text using Amazon Bedrock
Introduction
Amazon Bedrock is a AWS service that we use to build AI-Powered applications quickly without having to manage the servers or training the models ourselves.
Bedrock allows us to use pre-trained powerful AI models called Foundational Models (FMs) that are ready-to-use. It also provides easy integration with APIs and AWS services like SageMaker, which I have used for implementing RAG.
RAG (Retrieval-Augmented Generation) is a powerful technique that connects Large Language Models (LLMs) with external data knowledge sources to provide more accurate, context-aware, and factual answers. A challenge that we face for models is the limit on the amount of contextual information, which can be overcome by using RAG as it combines using embeddings to index the corpus of documents to build a Knowledge Base and uses a LLM to extract information from a subset of documents in that Knowledge Base.
The documents in the Knowledge Base are split into chunks (chunking), passed to an embedding model to obtain the embedding, and then the embedding combined with the original chunk of document and additional metadata are stored in the Vector Database (Knowledge Base).
The key issue of FMs is their tendency to hallucinate, therefore the context is important for: Question Answering (QA), contextual chatbots, etc. With Amazon Bedrock Knowledge Bases, we can securely connect FMs to data for RAG.
Building a Knowledge Base
To build a Knowledge Base, we need to take the new data and upload it to a data source. The data will be divided into chunks, which will then be passed to the embeddings model where the data will be embedded, and finally stored in the Vector Database.
We can choose any data source or multiple data source: Amazon S3, Confluence, Amazon Redshift, etc. Here, I have used S3.
Next, we upload data to the S3 bucket. We start the Ingestion process that includes taking the documents, make chunks, embeddings, sending the embeddings to the Vector Database. We have to keep syncing the S3 bucket and the Knowledge base if we add new data to the bucket. Storing the Knowledge Base Id is vital.
Data Source tells the Knowledge Base to pick the data from the source and acts as the bridge between the data and the Base.
Thus, the flow goes like this:
Workflow
We can ask queries and also ask for the specific chunks through which the LLM derived the information and gave us the answer from.
LLM giving the chunks with which it derived the answer
Retrieve API & RetrieveAndGenerate API in Bedrock
The RetrieveAndGenerate API combines both the retrieval and generation phases of RAG into a single operation. It queries a Bedrock Knowledge Base, retrieves relevant information, and then uses a designated Foundation Model (FM) to generate a coherent and context-aware response based on the retrieved results.
It returns a generated response from the FM, which is a human-readable answer derived from the retrieved context. It also provides the source attribution of the retrieved information. This API is designed for end-to-end RAG applications where you want Amazon Bedrock to manage both the retrieval and the generation of responses, providing a streamlined experience for building chatbots, question-answering systems, and other generative AI applications.
Here I've also used Custom Prompting and Streaming Response concepts.
Output of RetrieveAndGenerate API
On the other hand, The Retrieve API focuses solely on the retrieval phase of RAG. It allows you to query a Bedrock Knowledge Base (or other data sources like S3) and retrieve relevant information based on your query.
It returns the raw, retrieved content (e.g., document chunks, passages) that are deemed relevant to your query. It does not perform any summarization or generation of a human-like response.**** This API is suitable when you need to programmatically access the raw context from your knowledge base to build your own RAG pipeline, where you control the subsequent generation steps with a Large Language Model (LLM).
Here I also integrated LangChain, which is a framework that provides tools for chaining LLMs with data, APIs and external systems to create intelligent, context aware workflows.
Output of Retrieve API
RAGAS
RAGAS is an open-source library used to evaluate the performance of Retrieval-Augmented Generation (RAG) systems. It provides a framework and metrics to objectively assess AI applications built on RAG, ensuring they are accurate, relevant, and consistent. This helps developers improve their AI pipelines by measuring the quality of both the retrieval and generation components.
- Evaluates RAG pipelines: It provides a framework to evaluate systems that combine a language model with an external information retrieval mechanism.
- Uses various metrics: It offers a set of metrics to quantify performance, including:
- Faithfulness: Measures if the answer is factually consistent with the provided context.
- Answer Relevancy: Checks if the answer is relevant to the user's question.
- Context Precision/Recall: Evaluates how well the retrieval component finds the most relevant context from a large dataset.
- Enables continuous improvement: By providing a data-driven way to evaluate performance, Ragas helps create feedback loops to continuously improve AI applications.
- Offers flexibility: It supports custom metrics and integrates with popular frameworks like LangChain, making it easy to incorporate into existing workflows.
Output after RAGAS
Document-Level API
Document-Level API (DLA) is an API that allows for the direct ingestion and management of entire documents within a RAG system's knowledge base. Instead of requiring a full synchronization or complex data pipelines for updates, a DLA provides a streamlined way to add, update, or remove documents individually.
DLAs enable the immediate upload of new documents or datasets into the RAG system's knowledge base. This is particularly beneficial for applications dealing with frequently updated information, such as financial reports, news articles, or internal documentation.
DLAs simplify the process of maintaining the knowledge base by providing direct control over individual documents. This reduces the need for extensive data engineering and allows for more agile updates. They are also designed to seamlessly integrate with the broader RAG pipeline, ensuring that ingested documents are properly processed (chunked, embedded) and made available for retrieval by the language model.
Output after Document-Level API
Chunking
Advanced Chunking
Chunking in RAG is the process of splitting large documents into smaller, semantically meaningful segments or "chunks". This is a critical step for RAG systems because it ensures that relevant information can be retrieved efficiently and provided to the language model (LLM) without exceeding its context window limits. Well-executed chunking improves the accuracy, speed, and overall effectiveness of the RAG system.
There are multiple Chunking options:
- Fixed Chunking: Divides text into chunks of a predetermined, uniform size (e.g., a fixed number of characters, words, or tokens). Often includes an overlap between chunks to maintain context across boundaries.
- Semantic Chunking: Divides text into chunks based on the semantic meaning and coherence of the content. It aims to keep related information together in a single chunk, even if it means varying chunk sizes. This often involves using embeddings to group sentences or passages with high semantic similarity.
- Hierarchical Chunking: Organizes text into a hierarchical structure, reflecting the document's inherent organization (e.g., sections, subsections, paragraphs). This can involve recursive chunking, where a document is first split into larger units, which are then further split into smaller, more granular chunks based on predefined separators or semantic boundaries.
- Custom Chunking: Tailored chunking strategies designed to address specific needs or leverage unique characteristics of the data. This can involve combining elements of other strategies, incorporating domain-specific knowledge, or utilizing advanced techniques like agentic chunking (where an LLM determines optimal chunking based on content and structure).
Multiple types of Chunking
Autogenerated Filters
Auto-generated filter in RAG refers to the use of a large language model (LLM) to automatically extract constraints or criteria from a user's natural language query and convert them into structured metadata filters. These filters are then applied during the retrieval step to narrow down the search space and improve the relevance and accuracy of the retrieved documents.
How Auto-Generated Filters Work in RAG: Instead of a developer manually defining filters for every possible query, the RAG system leverages the LLM's natural language understanding capabilities to identify relevant filtering criteria. The process typically involves the following steps:
- Query Analysis: The user submits a natural language query, e.g., "What were the sales figures for Q4 2024 for the European market?".
- Filter Extraction: A dedicated component (often the LLM itself, fine-tuned for this task) analyzes the query and identifies key criteria, such as "Q4 2024" (date filter) and "European market" (region/topic filter).
- Structured Filter Generation: The identified criteria are converted into a structured query filter expression that the underlying vector database or search index can understand.
- Filtered Retrieval: The structured filter is applied during the search process. This pre-filtering step ensures that the vector similarity search is only performed on the subset of documents that match the specific criteria, significantly boosting retrieval speed and precision.
- Augmentation and Generation: The highly relevant, filtered documents are then combined with the original prompt to provide context for the final LLM, which generates a grounded and accurate response.
Autogenerated Filters
Metadata Filtering
Metadata filtering in RAG refines search results by using descriptive data (like dates, categories, authors) attached to text chunks to narrow down the context retrieved for an LLM, making responses more relevant, accurate, and efficient, especially for large datasets or complex queries by reducing noise and ensuring context aligns with specific user needs.
Metadata Filtering
Query Reformulation
Query Reformulation is used to increase accuracy for complex queries. It takes a complex input prompt and breaks it down into multiple sub-queries. They will then separately go through their own retrieval chunks. They can be pooled and ranked together before passing them to the FM to generate a response.
Query Reformulation
Re-ranking
Reranking in RAG is a crucial post-retrieval step that refines an initial set of documents by re-ordering them for relevance, ensuring only the most pertinent information is passed to the Large Language Model (LLM) for generating a high-quality, accurate response. It improves RAG system precision by using more sophisticated (but slower) cross-encoders or LLMs to deeply understand query-document semantic alignment, filtering out noise from the initial fast vector search.
Output after Re ranking
Combining Guardrails Contextual Grounding and Knowledge Bases
Guardrails with contextual grounding area set of safety and quality controls in Retrieval-Augmented Generation (RAG) applications designed to ensure that an AI model's responses are factually accurate based on provided source data and are relevant to the user's query. This helps prevent the model from generating incorrect or fabricated information, a problem known as hallucination.
In a RAG system, the large language model (LLM) retrieves relevant information from a knowledge base (e.g., enterprise documents) before generating a response. The guardrail's contextual grounding check then evaluates this response using specific metrics:
- Grounding Score: This metric measures whether the model's answer is factually supported by the information in the retrieved documents (the "reference source"). A high score indicates the response is well-anchored in the provided context, while a low score suggests a potential hallucination.
- Relevance Score: This metric assesses how closely the model's response relates to the user's original question. A high score means the AI is staying on topic, and a low score indicates an irrelevant answer.
Output after Guardrails Contextual Grounding
Audio and Video Data Prep using Amazon Bedrock Data Automation
Bedrock Data Automation (BDA) supercharges RAG by transforming unstructured, multimodal content (text, images, video, audio) into structured, usable data for richer, more accurate responses, acting as a powerful pre-processor for Bedrock Knowledge Bases to index multimodal insights, enabling RAG systems to answer complex questions from diverse data, not just text, with improved accuracy, explainability (visual grounding), and cost-effectiveness.
BDA automatically extracts content from multi-modal data. It streamlines the generation of valuable insights from unstructured multimodal content such as documents, images, audio, and videos through a unified multi-modal interference API.
Query the audio
Query the Video
Amazon Bedrock Knowledge Bases for Text and Images
Key Capabilities for Text & Images in RAG:
- Multimodal Data Ingestion: Automatically extracts text, tables, charts, and figures from documents (PDFs, etc.).
- Data Parsers: Use Bedrock Data Automation (Preview) or FMs (like Claude 3.5 Sonnet/Haiku) to parse complex layouts and extract insights.
- Semantic Embeddings: Generates embeddings for text and visual elements, storing them in a vector database (like Amazon Aurora, OpenSearch).
- Enhanced Retrieval: Enables querying for answers based on both textual and visual information.
- Source Attribution: Provides citations for retrieved visual and text data, increasing trust.
- Customization: Offers advanced chunking (semantic, hierarchical) and custom code via Lambda for full control.
- Managed RAG Workflow: Automates data ingestion, indexing, retrieval, and prompt augmentation.
Multimodal Data processing
Conclusion
In conclusion, leveraging a robust knowledge base alongside advanced techniques like RAG (Retrieval-Augmented Generation) can significantly enhance the performance of AI models. By evaluating RAGs with tools like RAGAS and optimizing processes through chunking, autogenerated filters, metadata filtering, and query reformulation, we can refine results. Re-ranking and guardrails ensure contextual grounding, while proper data preparation—across audio, video, image, and text formats using Amazon Bedrock—lays the foundation for more accurate, efficient, and scalable AI solutions.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article