Architecting Serverless RAG on AWS: From Ingestion to Retrieval
A practical architectural guide to building cost-effective, scalable Retrieval-Augmented Generation workflows using AWS Lambda, Amazon Bedrock, and OpenSearch Serverless.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Retrieval-Augmented Generation (RAG) grounds Large Language Models on domain-specific datasets without requiring continuous model fine-tuning. However, running dedicated compute clusters or always-on embedding pipelines frequently drives up infrastructure overhead for unpredictable workloads.
Adopting an event-driven serverless pattern on AWS enables automated vector ingestion, semantic querying, and managed inference while paying strictly for actual compute utilization.
---
### 1. System Architecture Overview
A production-grade serverless RAG setup decouples into two core pipelines: Knowledge Ingestion and Query & Inference.
* **Storage & Vectors:** Documents reside in Amazon S3, while vector embeddings and metadata index within an Amazon OpenSearch Serverless collection.
* **Compute:** AWS Lambda functions manage document parsing, vector indexing, and runtime orchestration.
* **Inference:** Amazon Bedrock provides managed endpoints for embedding generation (e.g., Titan Embeddings) and downstream text generation (e.g., Claude or Titan Text).
---
### 2. Vector Ingestion Pipeline
When new documents arrive in Amazon S3, an S3 Object Created event invokes an Ingestion Lambda. The handler reads the file, splits the text into manageable semantic chunks, generates vector embeddings using Amazon Bedrock, and indexes the payload into Amazon OpenSearch Serverless.
```python
import json
import boto3
bedrock = boto3.client("bedrock-runtime")
def get_embedding(text_chunk: str) -> list[float]:
response = bedrock.invoke_model(
modelId="amazon.titan-embed-text-v1",
contentType="application/json",
accept="application/json",
body=json.dumps({"inputText": text_chunk})
)
result = json.loads(response["body"].read())
return result["embedding"]
Key Ingestion Considerations:
- Chunking Strategy: Employ chunk sizes between 500 and 1,000 tokens with an overlap of 10–20% to prevent losing context across chunk boundaries.
- Batching: When uploading high-volume documents, batch vector indexing calls to OpenSearch Serverless to respect ingestion API rate limits.
3. Query Orchestration & k-NN Retrieval
During user queries, an orchestration Lambda converts the natural language question into an embedding vector, performs an approximate nearest neighbor (k-NN) vector search against OpenSearch Serverless, and assembles the prompt context.
1
2
3
4
5
6
7
8
9
def build_rag_prompt(user_query: str, retrieved_docs: list[str]) -> str:
context = "\n---\n".join(retrieved_docs)
return f"""You are an assistant. Answer the question using only the context provided below.
Context:
{context}
Question: {user_query}
Answer:"""
The assembled prompt is submitted to the Bedrock foundation model, ensuring answers remain strictly bounded to the retrieved documentation.
4. Operational Best Practices
- Cold Start Mitigation: Keep Lambda deployment bundles compact and configure appropriate memory settings (typically 1024 MB to 1792 MB) to balance CPU allocation and cold start latency.
- IAM Least Privilege: Separate roles for Ingestion and Query Lambdas. The query handler requires read-only access (aoss:ReadDocument) on the OpenSearch collection and invocation permissions on Bedrock models.
- Similarity Thresholding: Discard retrieved chunks whose cosine similarity score falls below an established minimum threshold to prevent irrelevant context injection into the generation step.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article