AWS Builder Center

Builder Center

Build a RAG API with FastAPI and Amazon Bedrock

Turn your own documents into a question-answering service, without managing any model infrastructure.

Why RAG?

Large language models are good at language but know nothing about your private data: your company PDFs, internal policies, or product manuals. Retrieval-Augmented Generation (RAG) fixes this by:
  1. Splitting your documents into chunks
  2. Converting each chunk into an embedding (a numeric vector)
  3. Finding the chunks most similar to a user's question
  4. Passing those chunks to an LLM as context so it answers from your data
Amazon Bedrock gives you both the embedding model and the LLM through one API, so there is no GPU setup and no model hosting.

Architecture

1
2
3
4
5
PDF/Text → Chunking → Titan Embeddings → Vector Store
↓
User Question → Embed → Similarity Search → Top-K Chunks
↓
Bedrock LLM (Converse API) → Answer

Prerequisites

  • An AWS account with Bedrock model access enabled for an embedding model (Amazon Titan Text Embeddings V2) and a text LLM, in your chosen region
  • Python 3.10+
  • AWS credentials configured locally (aws configure) with permission to call bedrock:InvokeModel
bash
1
pip install fastapi uvicorn boto3 numpy pypdf python-multipart

Step 1: Set up the Bedrock client

python
1
2
3
4
5
6
7
8
9
import json
import boto3
import numpy as np

REGION = "us-east-1"
EMBED_MODEL = "amazon.titan-embed-text-v2:0"
LLM_MODEL = "<your-bedrock-model-or-inference-profile-id>"

bedrock = boto3.client("bedrock-runtime", region_name=REGION)
Model IDs vary by region and change over time, so check the Bedrock console for the exact ID available to you.

Step 2: Create embeddings

python
1
2
3
4
5
6
7
8
9
def embed(text: str) -> np.ndarray:
body = json.dumps({
"inputText": text,
"dimensions": 512,
"normalize": True,
})
response = bedrock.invoke_model(modelId=EMBED_MODEL, body=body)
payload = json.loads(response["body"].read())
return np.array(payload["embedding"], dtype=np.float32)
Because the vectors are normalized, a simple dot product gives cosine similarity.

Step 3: Chunk and index documents

Chunking matters more than most beginners expect. Chunks that are too large dilute relevance, and chunks that are too small lose context. Overlap helps preserve meaning across boundaries.
python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from pypdf import PdfReader

def chunk_text(text: str, size: int = 800, overlap: int = 150):
chunks, start = [], 0
while start < len(text):
chunks.append(text[start:start + size])
start += size - overlap
return chunks

# Simple in-memory index (swap for a real vector DB in production)
INDEX = {"chunks": [], "vectors": None}

def add_document(text: str):
chunks = chunk_text(text)
vectors = np.vstack([embed(c) for c in chunks])
INDEX["chunks"].extend(chunks)
INDEX["vectors"] = (
vectors if INDEX["vectors"] is None
else np.vstack([INDEX["vectors"], vectors])
)

Step 4: Retrieve and generate

python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def retrieve(question: str, k: int = 4):
q = embed(question)
scores = INDEX["vectors"] @ q
top = np.argsort(scores)[::-1][:k]
return [INDEX["chunks"][i] for i in top]

def answer(question: str) -> str:
context = "\n\n---\n\n".join(retrieve(question))
prompt = (
"Answer the question using only the context below. "
"If the answer is not in the context, say you don't know.\n\n"
f"Context:\n{context}\n\nQuestion: {question}"
)
response = bedrock.converse(
modelId=LLM_MODEL,
messages=[{"role": "user", "content": [{"text": prompt}]}],
inferenceConfig={"maxTokens": 512, "temperature": 0.2},
)
return response["output"]["message"]["content"][0]["text"]
The Converse API gives you one consistent request format across Bedrock models, so you can swap LLMs by changing a single model ID.

Step 5: Expose it with FastAPI

python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from fastapi import FastAPI, UploadFile, File
from pydantic import BaseModel
import io

app = FastAPI(title="Bedrock RAG API")

class Query(BaseModel):
question: str

@app.post("/upload")
async def upload(file: UploadFile = File(...)):
data = await file.read()
reader = PdfReader(io.BytesIO(data))
text = "\n".join(page.extract_text() or "" for page in reader.pages)
add_document(text)
return {"status": "indexed", "chunks": len(INDEX["chunks"])}

@app.post("/ask")
def ask(query: Query):
if INDEX["vectors"] is None:
return {"answer": "No documents indexed yet."}
return {"answer": answer(query.question)}
Run it:
bash
1
uvicorn main:app --reload

Moving toward production

The in-memory index disappears when the server restarts. Before you ship, consider:
  • Persistent vector storage: use a vector database such as Amazon OpenSearch Service, Aurora PostgreSQL with pgvector, or a dedicated store like Qdrant.
  • Managed option: Knowledge Bases for Amazon Bedrock handles chunking, embedding, and retrieval for you, so you can skip Steps 3 and 4 if you don't need custom control.
  • Deployment: containerize the app and run it on AWS Lambda, ECS Fargate, or App Runner.
  • Security: use IAM roles instead of access keys, and scope permissions to specific Bedrock models.
  • Guardrails: add Amazon Bedrock Guardrails to filter harmful content and sensitive data.
  • Evaluation: test with real questions and check whether the retrieved chunks actually contain the answer. Most RAG failures are retrieval failures, not LLM failures.
  • Cost: embeddings are cheap, but LLM calls add up. Limit k and maxTokens, and cache repeated questions.

Common pitfalls

  • Answers ignore your documents: your prompt isn't constraining the model, or retrieval is returning irrelevant chunks. Print the retrieved context and inspect it.
  • AccessDeniedException: you haven't enabled model access in Bedrock for that region, or your IAM policy is missing bedrock:InvokeModel.
  • Poor results on tables or scanned PDFs: plain text extraction loses structure. Use OCR or a layout-aware parser first.

Conclusion

With about 60 lines of Python, you now have a working RAG backend: documents in, grounded answers out. From here, you can add a Streamlit front end, swap in a production vector store, or move to Knowledge Bases for a fully managed pipeline.
# rag# ai# amazon-bedrock# python
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article