AWS Builder Center
Building RAG twice for the AWS AI Practitioner exam: manual pipeline vs. Bedrock Knowledge Bases

Building RAG twice for the AWS AI Practitioner exam: manual pipeline vs. Bedrock Knowledge Bases

Building RAG twice for the AWS AI Practitioner exam: manual pipeline vs. Bedrock Knowledge Bases





Why I built the same RAG pipeline twice

While preparing for the AWS AI Practitioner (AIF-C01) exam, I implemented the same RAG architecture twice: first locally, with no framework (Ollama for embeddings and generation, word-based chunking with configurable overlap, cosine similarity via  numpy ), then ported the pipeline to Amazon Bedrock Knowledge Bases.
The goal was to isolate, step by step, what actually reduces hallucination in a RAG pipeline. Two tests made this concrete in a way theory alone hadn't.

The experiment

I asked the same question — "when should you use RAG instead of fine-tuning?" — with no retrieved context, against two models of different capability:
  • Llama 3.2 (3B, local via Ollama) replied in Italian, unprompted, defining RAG as "Resizing and Padding" and producing a PyTorch snippet with a parameter that doesn't exist in the Transformers API.
  • Amazon Nova Micro (via Bedrock) didn't make that kind of gross error, but showed a subtler conceptual slip: it described RAG as a technique that "combines" with fine-tuning, when in practice they're alternative approaches to different problems — knowledge adaptation (RAG) versus behavior adaptation (fine-tuning).

The pattern

Smaller models tend to hallucinate obviously, including surface-level inconsistencies like switching languages mid-answer. Larger models tend to hallucinate subtly, with conceptual conflations plausible enough to mislead someone still learning the material. Grounding via retrieval fixed both cases equally, regardless of the base model's capability — which reinforces why RAG should be treated as general hallucination mitigation, not a workaround for weak models.

The cost lesson

Migrating to Bedrock surfaced a cost lesson that's probably more relevant to newcomers than to teams already running Bedrock in production: the console's "quick create" flow nudges you toward Amazon OpenSearch Serverless as the vector store, which carries a 4 OCU floor (~$700/month) billed regardless of usage.
I used the Bedrock Managed Knowledge Base instead (released June 2026), which abstracts the vector store choice away entirely and bills based on actual usage — indexed storage plus retrieve calls.


Curious whether anyone here has run the Managed Knowledge Base in production and how the cost compares to a self-managed S3 Vectors setup at larger scale — with just a handful of documents, the difference is negligible, but I'd like to understand the breakeven point better.

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article