Context-Aware AI Applications with RAG Workshop

Building trustworthy AI systems requires more than just a powerful model. It requires a robust framework that can retrieve accurate information, restrict hallucinations, validate outputs, and ensure that responses align with enterprise standards. In this workshop, we explore how Amazon Bedrock enables this through a combination of Knowledge Bases (for Retrieval-Augmented Generation) and Guardrails (for safety and contextual grounding).

srini_03

Published Dec 3, 2025

Last modified Dec 3, 2025

1. Introduction: Why LLM Hallucinations Are a Critical Problem

LLMs have become indispensable in a wide variety of applications: customer support agents, search engines, chatbots, documentation assistants, game knowledge engines, and enterprise automation workflows. Despite their power, all LLMs—including top-tier models—are prone to hallucinations.

What is a Hallucination?

A hallucination occurs when a model:

Invents facts not present in any context
Makes up numbers, rules, or events
Confidently states incorrect or fabricated information
Fills gaps with imaginative guesses rather than real data

This is dangerous for production environments.

Why Hallucinations Happen

LLMs are predictive text machines: they generate likely continuations, not guaranteed truths.
Without grounding, they have no way to verify information.
When asked about specific domains (finance, healthcare, gaming rules, law), they might guess if not guided by trusted data.

Why This Matters for Enterprises

Hallucinations can lead to:

Incorrect advice for customers
Compliance violations
Financial risk
Legal exposure
Broken user trust

Therefore, modern enterprises cannot rely solely on raw LLM output. They need systems that enforce correctness, verify factual grounding, and tightly control model behavior.

2. Step 1: Building the Knowledge Base with RAG

Amazon Bedrock Knowledge Bases implement RAG without requiring developers to build their own vector databases, pipeline components, or chunking strategies manually. The system ingests documents, converts them into embeddings, stores them in a vector database, and retrieves context efficiently.

In this example, we use the Dungeons & Dragons (D&D) System Reference Document (SRD)—a structured set of rules ideal for evaluating factual grounding.

How the Knowledge Base Works (Deep Explanation)

2.1 Data Source

The raw dataset (SRD PDF) is uploaded to Amazon S3, ensuring:

Durability
Versioning support
Easy pipeline integration
IAM-based access control

2.2 Vector Database Layer

Amazon OpenSearch Serverless stores the embeddings.
This ensures:

Ultra-fast semantic search
Low-latency retrieval
Automatic scaling
Zero database maintenance

2.3 Embedding Model

The Titan Embeddings Text V2 model converts passages into dense vectors.
These vectors encode meaning, not keywords.

Semantic retrieval helps answer questions like:

“What abilities do High Elves have?”

…even if the wording in the document is different.

2.4 Chunking Strategy

The PDF is chunked into sections:

300 tokens per chunk
20% overlap between chunks
Fixed size ensures consistency

This provides:

Enough context in each chunk
No loss of important details
Better grounding during answer generation

Knowledge Base Ingestion Summary

Once indexing is complete, you get a status confirmation:

1
2
3
4
5
6
7
{
  "status": "COMPLETE",
  "statistics": {
    "numberOfDocumentsScanned": 1,
    "numberOfNewDocumentsIndexed": 1
  }
}

The system is now ready to support RAG queries.

3. Step 2: Creating the Guardrail for Contextual Grounding

RAG alone solves half the problem. It retrieves data but does not prevent the model from:

Adding extra facts
Guessing
Misinterpreting context

This is where Amazon Bedrock Guardrails come in.

Guardrails evaluate each response using numeric filters and block any output that fails validation.

The Two Core Filters: How They Work

1. Grounding Filter

Checks:
Is the model’s answer factually tied to the retrieved documents?

A grounding score of 0.5 means at least 50% confidence that the response is supported by the supplied context.

2. Relevance Filter

Checks:
Is the response actually related to what the user asked?

A relevance threshold of 0.8 requires the model to be highly focused on the user’s query.

Together, these filters prevent:

Hallucinated reasoning
Irrelevant responses
Creative fabrications
“Educated guesses”

Why Threshold Tuning Is Important

Different industries have different risk tolerance levels:

Industry	Recommended Grounding	Recommended Relevance	Reason
Healthcare	0.9	0.9	Zero tolerance for mistakes
Finance	0.8+	0.8+	Must be fully compliant
Gaming / SRD	0.5	0.8	Flexible but must be factual
Creative	0.3	0.5	Allows imagination

This allows developers to customize AI behavior to business needs.

Guardrail Code Example

1
2
3
4
5
6
7
8
9
10
11
12
response = bedrock.create_guardrail(
    name="contextual-grounding-guardrail-{}",
    description="D&D Guardrail",
    contextualGroundingPolicyConfig={
        'filtersConfig': [
            {'type': 'GROUNDING', 'threshold': 0.5},
            {'type': 'RELEVANCE', 'threshold': 0.8},
        ]
    },
    blockedInputMessaging="Sorry, I can not respond to this.",
    blockedOutputsMessaging="Sorry, I can not respond to this.",
)

Guardrails now act as a real-time safety net over the LLM.

4. Step 3: Contextual Grounding in Real Action

Two APIs work together:

1. Retrieval

bedrock-agent-runtime.retrieve pulls relevant chunks from the Knowledge Base.

2. Generation

bedrock-runtime.converse uses Claude 3 Sonnet, enriched with retrieved chunks, then validates results via Guardrails.

Workflow

User asks question
RAG retrieves context
LLM generates answer
Guardrails evaluate grounding + relevance
If approved → delivered
If rejected → safe fallback response

This ensures high reliability.

Example 1: Grounded Query

User: “What are High Elves?”

Response Characteristics:

Accurate summary
Pulled directly from SRD
No invention or creative additions

Scores:

Grounding: 0.79
Relevance: 1.0

Outcome:

Response is allowed.

This proves RAG + Guardrails work correctly for valid questions.

Example 2: Irrelevant Question (Testing Hallucination Prevention)

User: “Where should the elves go if they arrive in Paris?”

SRD has no mention of Paris.

Expected Behavior:

The model must:

Not imagine answers
Not create new lore
Not mix real world with fantasy unless allowed

Actual Response:

“The passage does not provide any specific information about where elves should go if they arrive in Paris…”

Scores:

Grounding: 0.93
Relevance: 1.0

Outcome:

Response is allowed because it is fully grounded (the content says nothing about Paris).

This demonstrates that Guardrails prevent hallucinations without unnecessarily rejecting harmless queries.

5. Expanded Conclusion: Why This Architecture Is Transformative

Combining RAG + Guardrails creates a highly reliable, enterprise-ready AI stack. This solution goes far beyond retrieval and generation — it creates a controlled intelligence system with predictable and explainable behavior.

Key Benefits (Expanded)

✔ 1. Maximum Accuracy Through Verified Context

Guardrails force the LLM to stick to the source material, dramatically reducing hallucinations.

✔ 2. Consistent and Reproducible Results

Because every output is validated, results remain consistent across:

Different users
Different queries
Different environments

✔ 3. Enterprise-Grade Trust and Compliance

Confidence scores act as:

Audit logs
Explainability metadata
Compliance evidence

This is important for industries governed by regulations.

✔ 4. Operational Efficiency

Companies avoid the need to:

Build custom hallucination filters
Fine-tune large models constantly
Manually validate responses

The platform does it automatically.

✔ 5. Scalable Knowledge Management

Knowledge Bases allow:

Continuous ingestion
Automatic re-indexing
Support for millions of documents

No infrastructure management is required.

✔ 6. Strong User Experience

When context doesn’t exist, the response is neutral and factual:

“The document does not contain that information.”

This clarity builds trust and avoids confusion.

Real-World Use Cases (New Section)

1. Customer Support Automation

Providing answers strictly from product manuals or official policies.

2. Legal Assistants

Summarizing sections only from provided legal documents.

3. Healthcare Knowledge Systems

Offering medical guidance only from approved medical literature.

4. Financial Bots

Explaining investment products based solely on official documentation.

5. Game Databases

Responding to player queries based on official rulebooks.

Architectural Perspective

A modern AI application using this stack includes:

LLM (Claude 3 Sonnet)
Knowledge Base for Retrieval
Vector Database (OpenSearch Serverless)
Guardrails for Output Validation
API Gateway or Bedrock Runtime Layer

This helps deliver:

Reliability
Explainability
Full governance control

# ai # aws-builders # workshop # knowledge-bases-for-bedrock # aws-workshops

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Context-Aware AI Applications with RAG Workshop

srini_03

1. Introduction: Why LLM Hallucinations Are a Critical Problem

What is a Hallucination?

Why Hallucinations Happen

Why This Matters for Enterprises

2. Step 1: Building the Knowledge Base with RAG

How the Knowledge Base Works (Deep Explanation)

2.1 Data Source

2.2 Vector Database Layer

2.3 Embedding Model

2.4 Chunking Strategy

Knowledge Base Ingestion Summary

3. Step 2: Creating the Guardrail for Contextual Grounding

The Two Core Filters: How They Work

1. Grounding Filter

2. Relevance Filter

Why Threshold Tuning Is Important

Guardrail Code Example

4. Step 3: Contextual Grounding in Real Action

1. Retrieval

2. Generation

Workflow

Example 1: Grounded Query

Response Characteristics:

Scores:

Outcome:

Example 2: Irrelevant Question (Testing Hallucination Prevention)

Expected Behavior:

Actual Response:

Scores:

Outcome:

5. Expanded Conclusion: Why This Architecture Is Transformative

Key Benefits (Expanded)

✔ 1. Maximum Accuracy Through Verified Context

✔ 2. Consistent and Reproducible Results

✔ 3. Enterprise-Grade Trust and Compliance

✔ 4. Operational Efficiency

✔ 5. Scalable Knowledge Management

✔ 6. Strong User Experience

Real-World Use Cases (New Section)

1. Customer Support Automation

2. Legal Assistants

3. Healthcare Knowledge Systems

4. Financial Bots

5. Game Databases

Architectural Perspective