Retrieval-Augmented Generation on AWS: Skip the Vector Database
Building a RAG AI with Amazon Bedrock managed knolwedge base.
Here’s what nobody tells you before you build your first RAG pipeline: the hard part was never the language model. It’s the plumbing around it. RAG (Retrieval-Augmented Generation) is a way of answering questions using your own documents instead of whatever a language model happened to memorize during training. Ask a plain model about your company’s return policy and it either admits it doesn’t know, or invents something plausible and wrong. RAG fixes that by pulling the relevant paragraph from your actual policy document first, then handing that paragraph to the model along with the question. The model answers from what it was just given, not from memory.
The traditional way to build this involves three separate systems, and each one has its own way of breaking. First, a vector store: a specialized database that finds text by meaning rather than exact wording. Second, an embedding model, which converts a chunk of text into a list of numbers (a “vector”) that the vector store can search. Third, chunking logic: code you write to split a long document into smaller pieces, because feeding an entire book into a model at once doesn’t work well. You provision all three, wire them together, and keep them in sync every time someone edits the source document.
Amazon Bedrock’s Managed Knowledge Base collapses all three into one thing you don’t have to configure. You upload a document to S3 (Amazon’s file storage service), point a knowledge base at that S3 location, and Bedrock handles chunking, embedding, and storage automatically. No vector database to name, no embedding model to pick. If you’re building this on AWS, that’s the version to start with, and the rest of this article shows you how, end to end.
The shape of the system
Three pieces, four steps. You upload a document to an S3 bucket. A knowledge base “syncs” against that bucket: it reads the document, breaks it into chunks, and converts each chunk into a vector, automatically, using a model Bedrock manages for you. Then, when a question comes in, a small piece of your own code, an AWS Lambda function that AWS runs for you on demand without you managing a server, calls to fetch the matching chunks, and calls a second API, to turn those chunks into an actual written answer.
One thing worth knowing before you build this: Bedrock offers a convenience API called RetrieveAndGenerate that normally handles the whole pipeline (search plus answer) in a single call. Managed Knowledge Bases don’t support it. You get Retrieve, which returns matching chunks and nothing else, and you write the “turn these chunks into an answer” step yourself. That sounds like a downgrade. It isn’t: writing that step yourself means you control exactly what happens when nothing relevant is found, instead of trusting a black-box default you can’t see inside.
Building it: step by step
Step 1: Upload a document to S3
Open the S3 console and create a bucket. Pick a globally unique name and the AWS Region you plan to use throughout, for example US East (N. Virginia). Leave the default security settings (Block Public Access on) and create the bucket.
Then upload the document you want your application to answer questions about. It can sit at the top level of the bucket or inside a folder; either works.
Step 2: Choose a model
Open the Bedrock console, go to the Model catalog. Pick a generation model, the one that will actually write your answers. If this is the first time your AWS account has used a model from Anthropic, you’ll be asked to fill out a short use-case form, including a website URL. If you don’t have a company site, a GitHub or LinkedIn profile works fine here; this is a one-time step per account.
While you’re on the model’s page, check whether it requires something called “cross-Region inference.” If it does, note the model’s inference profile ID (something like us.anthropic.claude-sonnet-4–6) because you’ll need that exact value later, not the plain model name.
Step 3: Create the knowledge base
In the Bedrock console, go to Knowledge Bases and choose Create. When asked for a type, choose Managed Knowledge Base (sometimes labeled “Serverless”). This is what skips the vector store setup entirely: there’s no embedding model to pick and nothing else to configure. Give it a name, let the console create its permissions role automatically, and point its data source at the S3 bucket from Step 1.
Once it’s created, it syncs automatically with the data source. This is the step that actually reads your document, breaks it into chunks, and indexes it. For a short document this finishes in seconds; for something long, it can take several minutes. Check the sync status before assuming something’s wrong.
Step 4: Write and deploy the Lambda function
This is the one piece of code in the whole project. Create a new Lambda function in the console, using Python, and paste in the code from the next section. Before it will work, three settings need attention:
- The package. The code below uses newer AWS SDK functionality, including “managedSearchConfiguration”: { }. The version of boto3 included in Lambda’s Python runtime may not support the required API features yet. AWS updates the runtime’s built-in SDK periodically. Install a compatible current copy of boto3 yourself and bundle it with your code.
1
2
3
4
mkdir lambda-package && cd lambda-package
pip install boto3 - target . - upgrade
cp /path/to/lambda_function.py .
zip -r ../function.zipNOTE: Run that last command from inside the lambda-package folder, targeting “.”. If you zip the folder itself instead of its contents, your code and dependencies will be nested one level too deep, and Lambda may not be able to find your handler.
Upload the resulting function.zip as the lambda deployment package.
- The timeout. Lambda functions have a maximum run time, and the default timeout is three seconds. A Retrieve call followed by a Converse call may take longer than that. In your function’s Configuration tab, increase the timeout to thirty seconds before you test anything. Otherwise, the function may time out before it has completed the requests.
- The environment variables. Still in Configuration, add two: KNOWLEDGE_BASE_ID (found on your knowledge base’s overview page) and MODEL_ARN (the inference profile ARN from Step 2, if your selected model requires one).
Finally, give the function permission to actually call Bedrock. Attach a policy like this to its execution role:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:Retrieve",
"bedrock:RetrieveAndGenerate",
"bedrock:InvokeModel"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}Step 5: Expose it and test it
In your function’s Configuration tab, create a Function URL and set its authentication to NONE, which gives you a plain HTTPS address to test against, no signing required. Copy that URL, then run:
1
2
3
curl -X POST "https://<your-function-url>.lambda-url.us-east-1.on.aws/" \
-H "Content-Type: application/json" \
-d '{"question": "<your question here>"}'If it works, you’ll get back a JSON response with an answer grounded in your uploaded document.
The Lambda function, in full
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import json
import boto3
import os
REGION = os.environ.get("AWS_REGION", "us-east-1")
KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
MODEL_ARN = os.environ["MODEL_ARN"] # generation model, e.g. Claude
agent_client = boto3.client("bedrock-agent-runtime", region_name=REGION)
runtime_client = boto3.client("bedrock-runtime", region_name=REGION)
def lambda_handler(event, context):
body = event.get("body")
payload = json.loads(body) if isinstance(body, str) else event
question = payload.get("question")
if not question:
return {"statusCode": 400, "body": json.dumps({"error": "Missing 'question'"})}
# Step 1: Retrieve chunks — managed KBs use managedSearchConfiguration, not vectorSearchConfiguration
retrieve_response = agent_client.retrieve(
knowledgeBaseId=KNOWLEDGE_BASE_ID,
retrievalQuery={"text": question},
retrievalConfiguration={
"managedSearchConfiguration": {}
},
)
chunks = retrieve_response["retrievalResults"]
context_text = "\n\n".join(c["content"]["text"] for c in chunks)
# Step 2: Generate an answer yourself using the retrieved context
prompt = f"""Answer the question using only the context below. If the context doesn't contain the answer, say so.
Context:
{context_text}
Question: {question}"""
converse_response = runtime_client.converse(
modelId=MODEL_ARN,
messages=[{"role": "user", "content": [{"text": prompt}]}],
)
answer = converse_response["output"]["message"]["content"][0]["text"]
return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps({
"answer": answer,
"source_chunk_count": len(chunks),
}),
}Testing it honestly
Once it’s running, don’t just ask one question and call it done. Ask something with a clear, checkable answer in your document first, so you can confirm the whole pipeline is actually connected end to end. Then ask the same question again with different wording, to see whether the system is matching on meaning or just got lucky matching the same words. Then ask something your document never mentions at all.
That last test matters the most: a system that says “I don’t have that information” is working correctly. A system that confidently makes something up has failed at the one job RAG exists to do, even if every other test passed.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article