Production RAG: Implementing Hybrid Search, RRF, and Semantic Chunking on AWS
While a basic Retrieval-Augmented Generation (RAG) setup using simple text chunking and vector search works fine for basic demo applications, it often fails in enterprise production environments. Standard vector searches struggle to find exact part numbers, product IDs, or specific names, while naive character-based text slicing often cuts sentences in half, stripping out vital context.
- Moving Beyond Character Limits: Semantic Chunking
Fixed-size chunking (e.g., cutting text every 512 characters) introduces noise. If an important rule spans across two chunks, the vector embedding loses its thematic cohesion.
Semantic Chunking uses an embedding model to evaluate relationships between sentences. It calculates the statistical semantic distance between sentence sequences and splits the document only when a sharp shift in the topic occurs.
Implementation with AWS Lambda and Bedrock
When an enterprise document hits an Amazon S3 bucket, an asynchronous data preparation pipeline processes it using the following logic:
python
import numpy as np
def calculate_cosine_distance(embedding_a, embedding_b):
"""Measures the divergence between two sentence embeddings."""
return 1 - (np.dot(embedding_a, embedding_b) / (np.linalg.norm(embedding_a) * np.linalg.norm(embedding_b)))
"""Measures the divergence between two sentence embeddings."""
return 1 - (np.dot(embedding_a, embedding_b) / (np.linalg.norm(embedding_a) * np.linalg.norm(embedding_b)))
def split_document_semantically(sentences, embeddings, distance_threshold=0.3):
"""Groups sentences into cohesive chunks based on thematic continuity."""
chunks = []
current_chunk = []
"""Groups sentences into cohesive chunks based on thematic continuity."""
chunks = []
current_chunk = []
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
for i in range(len(sentences) - 1):
current_chunk.append(sentences[i])
# Calculate semantic distance to the next sentence
distance = calculate_cosine_distance(embeddings[i], embeddings[i+1])
# If the topic changes drastically, cut the chunk here
if distance > distance_threshold:
chunks.append(" ".join(current_chunk))
current_chunk = []
if current_chunk:
current_chunk.append(sentences[-1])
chunks.append(" ".join(current_chunk))
return chunksUse code with caution.
By utilizing amazon.titan-embed-text-v2 via Bedrock to compute these sentence-level arrays, your ingested documents are segmented into clean, contextual paragraphs.
2. Overcoming Vector Blind Spots: Hybrid Search
Vector search (k-Nearest Neighbors) is excellent at capturing abstract concepts and intent, but it is notoriously poor at finding precise tokens. For instance, querying a database for product code "X-942-B" might return documents about entirely different items if they share a similar abstract description.
Hybrid Search solves this by executing two queries simultaneously within Amazon OpenSearch Serverless:
By utilizing amazon.titan-embed-text-v2 via Bedrock to compute these sentence-level arrays, your ingested documents are segmented into clean, contextual paragraphs.
2. Overcoming Vector Blind Spots: Hybrid Search
Vector search (k-Nearest Neighbors) is excellent at capturing abstract concepts and intent, but it is notoriously poor at finding precise tokens. For instance, querying a database for product code "X-942-B" might return documents about entirely different items if they share a similar abstract description.
Hybrid Search solves this by executing two queries simultaneously within Amazon OpenSearch Serverless:
- A Neural (Vector) Search: To match the overall conceptual meaning.
- A Keyword (Lexical) Search: Using the BM25 algorithm to look for exact keyword matches, SKU numbers, and names.
OpenSearch Serverless Hybrid Query Structure
You can execute both search methodologies in a single payload inside OpenSearch:
json
{
"size": 5,
"query": {
"hybrid": {
"queries": [
{
"match": {
"text_content": "Find policy updates for server configuration X-942-B"
}
},
{
"knn": {
"vector_field": {
"vector": [0.123, -0.456, 0.789, "..."],
"k": 5
}
}
}
]
}
}
}
Use code with caution. - Merging the Streams: Reciprocal Rank Fusion (RRF)
Executing a hybrid search yields two separate lists of matching documents, each scored by completely different mathematical metrics (BM25 keyword relevance vs. Vector cosine similarity scores). You cannot simply add these scores together.
Reciprocal Rank Fusion (RRF) normalizes the results. It completely discards the raw scores and instead looks exclusively at the relative rank (position) of a document within both individual result lists.
The algorithm uses a constant factor (k, usually set to 60) to score each document:
(RRF_Score(d)=\sum {m\in M}\frac{1}{k+r{m}(d)})
Where M represents the search methods (lexical and vector), and (r_m(d)) is the rank position of document d in that specific search method.
Programmatic RRF Ranking in Python
python
def reciprocal_rank_fusion(lexical_results, vector_results, k=60):
"""Merges and reranks hybrid search results using RRF scores."""
rrf_scores = {}# Process lexical keyword rankingsfor rank, doc_id in enumerate(lexical_results):
rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + (1.0 / (k + (rank + 1)))# Process vector rankingsfor rank, doc_id in enumerate(vector_results):
rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + (1.0 / (k + (rank + 1)))# Sort documents by their combined RRF score descendingsorted_docs = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)
return sorted_docs
Use code with caution.
Amazon OpenSearch Serverless offers built-in, native support for RRF orchestration inside its engine pipelines. This allows you to automatically receive a single, perfectly unified list of high-relevance chunks without needing custom application-side math scripts.
Summary Architecture for Enterprise Accuracy
By implementing this advanced tier, the runtime flow transforms:
┌──> Lexical Search (BM25) ──┐
User Query ───────┼──> Vector Search (k-NN) ──┼──> Native RRF Pipeline ──> Grounded LLM Prompt
└──> [Embeddings Engine] ───┘
Grounding an LLM (such as Anthropic Claude 3.5 Sonnet on Bedrock) with context derived from a semantic hybrid engine reduces model hallucinations to near zero. It balances conceptual intelligence with precise document filtering, producing an incredibly reliable interface for complex enterprise datasets.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article