RAG Was Painful Until Now: How Amazon Bedrock Managed Knowledge Base Eliminates the Plumbing
Amazon Bedrock Managed Knowledge Base is a fully managed RAG service that handles ingestion, vector storage, and retrieval optimization. Learn how native connectors, Smart Parsing, and Agentic Retriever eliminate the infrastructure complexity of building enterprise RAG pipelines.
Every RAG Pipeline Starts Simple and Ends in Infrastructure Chaos
You want your AI application to answer questions using your company's documents. Simple enough, right?
Then reality hits. You need to:
- Choose an embedding model and deploy it
- Set up a vector database (OpenSearch, Pinecone, pgvector?)
- Build an ingestion pipeline that handles PDFs, Word docs, HTML, and Confluence pages
- Chunk your documents (fixed size? semantic? overlap?)
- Configure a re-ranking model to improve relevance
- Build a retrieval API that your agent can call
- Monitor ingestion failures, stale indexes, and query latency
- Scale all of this when your document corpus grows from 1,000 to 1,000,000 files
By the time you are done, you have built and maintained six infrastructure components just to answer: "What does our refund policy say about international orders?"
Amazon Bedrock Managed Knowledge Base, announced as generally available at AWS Summit NYC on June 17, 2026, eliminates this entire stack. You connect your data sources, and the service handles everything else.
What Is Amazon Bedrock Managed Knowledge Base?
It is a fully managed RAG service that connects to enterprise data sources like SharePoint, Confluence, Google Drive, OneDrive, Amazon S3, and Web Crawler. It automatically handles ingestion, vector storage, and retrieval optimization.
According to the official AWS announcement, it abstracts away "the multiple infrastructure components developers traditionally have to assemble and maintain themselves (storage, retrieval, embeddings, re-ranking, and foundation model selection) into a single managed primitive."
No vector database to provision. No embedding pipeline to manage. No chunking strategy to tune.
The Three Features That Change Everything
1. Native Data Connectors
Instead of building custom ETL pipelines for every data source, you configure connectors directly:
| Data Source | What It Does |
|---|---|
| Amazon S3 | Monitors buckets for new/updated documents |
| SharePoint Online | Syncs document libraries and site pages |
| Confluence | Indexes spaces, pages, and attachments |
| Google Drive | Connects shared drives and folders |
| OneDrive | Syncs personal and shared files |
| Web Crawler | Follows links and indexes web content |
Each connector handles authentication, incremental sync, and format conversion automatically. When a document updates in SharePoint, the knowledge base re-indexes it without manual intervention.
2. Smart Parsing
This is where the real magic happens. Traditional RAG pipelines struggle with complex documents because they treat everything as flat text. A table becomes garbled rows. A chart loses all meaning. Headers and footers repeat on every chunk.
Smart Parsing understands document structure:
- Tables are parsed as structured data, preserving row and column relationships
- Images with text get OCR treatment
- Headers and sections maintain hierarchy so chunks carry context
- Multi-format support handles PDF, DOCX, PPTX, HTML, Markdown, and more
The result: when your agent retrieves a chunk about pricing tiers, it gets the actual pricing table with rows and columns intact, not a jumbled paragraph of numbers.
3. Agentic Retriever for Multi-Step Queries
This is the capability that moves RAG from "search" to "reasoning."
Traditional RAG retrieves documents and passes them to a model. It works for simple questions like "What is our vacation policy?" but fails on complex ones like "Compare our Q1 and Q3 revenue performance and explain what drove the difference."
The Agentic Retriever handles complex multi-hop queries by automatically orchestrating:
- Query planning: Breaking a complex question into sub-queries
- Interim response evaluation: Checking whether retrieved documents actually answer the question
- Re-ranking: Reordering results based on actual relevance to the decomposed sub-queries
According to the AWS documentation, it supports "hybrid search, document ranking, and agentic retrieval that automatically orchestrates query planning, interim response evaluation, and re-ranking for complex multi-hop queries."
Real World Example: A Customer Support Agent
Here is a practical scenario showing the difference.
The Setup:
- 50,000 support articles in Confluence
- 10,000 product spec PDFs in S3
- 3,000 FAQ pages on a public website
Without Managed Knowledge Base:
You need:
- A Lambda function to poll Confluence for updates (custom code)
- An S3 event pipeline to trigger reprocessing when PDFs change
- A web scraper on a schedule for FAQ pages
- A chunking service (probably another Lambda)
- An OpenSearch Serverless cluster for vector storage
- An embedding model endpoint (SageMaker or Bedrock)
- A query API that handles retrieval and re-ranking
- A monitoring dashboard for all of the above
Estimated setup time: 4 to 6 weeks for an experienced team.
With Managed Knowledge Base:
You configure three data connectors (Confluence, S3, Web Crawler), set up sync schedules, and start querying. The service handles parsing, chunking, embedding, storage, and retrieval.
Setup time: under one hour.
The Query:
A customer asks: "I bought the Enterprise plan last March. The contract says I get priority support with 4 hour response SLA. My last three tickets took over 24 hours. What is my compensation according to the SLA terms?"
The Agentic Retriever:
- Decomposes into sub-queries: (a) Enterprise plan SLA terms, (b) SLA violation compensation policy, (c) Priority support definition
- Retrieves relevant chunks for each sub-query independently
- Evaluates whether the retrieved content actually answers each sub-question
- Re-ranks and consolidates the final context for the modelThe agent responds with the specific SLA clause, the compensation formula, and the customer's entitlement, all grounded in actual documents.
Integration with AgentCore Gateway
Managed Knowledge Base plugs directly into Amazon Bedrock AgentCore as a native Gateway connector. According to the documentation:
"Amazon Bedrock AgentCore exposes a managed knowledge base as a native Gateway connector. You attach it to your AgentCore Gateway and your agents discover and query it with standard Model Context Protocol (MCP) calls, with no custom retrieval integration to build."
This means:
- Your agents discover the knowledge base automatically through MCP
- No custom retrieval code needed
- Permissions are enforced at the Gateway level
- Multiple agents can share the same knowledge base
Optimized Defaults: No Tuning Required
One of the most developer-friendly decisions is the optimized defaults approach. The service pre-configures:
- Embedding model selection based on your content type
- Chunk size and overlap based on document structure
- Re-ranking model for result relevance
- Hybrid search combining keyword and semantic matching
You can override any of these if you have specific requirements, but most teams ship to production with the defaults and never touch them.
How to Get Started
Step 1: Create a Knowledge Base
In the Amazon Bedrock console:
- Navigate to Knowledge Bases
- Click "Create knowledge base"
- Name it and select your IAM role
Step 2: Add Data Sources
Configure one or more connectors:
- For S3: Point to your bucket and prefix
- For Confluence: Provide the site URL, space keys, and OAuth credentials
- For SharePoint: Connect via app registration with site and drive IDs
- For web content: Specify seed URLs and crawl depth
Step 3: Start Syncing
Trigger the first sync. The service:
- Connects to your data source
- Downloads and parses documents using Smart Parsing
- Generates embeddings
- Stores vectors and metadata
- Makes everything queryable
Step 4: Query or Connect to an Agent
Direct API call:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
client = boto3.client('bedrock-agent-runtime')
response = client.retrieve_and_generate(
input={'text': 'What is our refund policy for international orders?'},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': 'YOUR_KB_ID',
'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-20250514-v1:0'
}
}
)
print(response['output']['text'])As an AgentCore Gateway connector:
Attach the knowledge base to your AgentCore Gateway. Agents discover and query it through MCP calls automatically.
Attach the knowledge base to your AgentCore Gateway. Agents discover and query it through MCP calls automatically.
Cost Considerations
Managed Knowledge Base pricing covers:
- Data ingestion: Per page parsed and embedded
- Storage: Vector storage for your indexed content
- Queries: Per retrieval requestThere is no separate charge for the embedding model, vector database, or orchestration layer. You pay for usage, not infrastructure.
Compare this to self-managed RAG where you pay separately for: OpenSearch Serverless (minimum 2 OCUs = ~$700/month), SageMaker embedding endpoints (~$200/month for a small model), Lambda invocations, S3 storage, and engineering time to maintain it all.
When to Use Managed Knowledge Base vs. Building Your Own
| Scenario | Use Managed KB | Build Your Own |
|---|---|---|
| Standard document Q&A | Yes | No |
| Multi-source enterprise RAG | Yes | No |
| Custom embedding models (fine-tuned) | No | Yes |
| Sub-10ms latency requirements | No | Yes |
| Documents change frequently | Yes | Depends |
| Need full control over chunking logic | No | Yes |
| Team has no ML infrastructure experience | Yes | No |
For 90% of enterprise RAG use cases, Managed Knowledge Base is the right choice. You build your own only when you have highly specialized retrieval requirements that the managed defaults cannot satisfy.
References
- Introducing Amazon Bedrock Managed Knowledge Base for faster, more accurate enterprise AI applications (June 17, 2026)
- Amazon Bedrock Managed Knowledge Base is now generally available (June 17, 2026)
- Amazon Bedrock Knowledge Bases product page
- Amazon Bedrock Managed Knowledge Bases as Connector Target (AgentCore Gateway)
- Top announcements of the AWS Summit in New York, 2026 (June 17, 2026)
- New in Amazon Bedrock AgentCore: Build agents with broader knowledge and continuous learning (June 17, 2026)
- Amazon Bedrock Knowledge Bases documentation
About the Authors
Sekh Mohammad Warish is a Senior Technical Account Manager at AWS, supporting customers across SAP, GenAI, and cloud modernization. With 120+ IT certifications including 12 AWS certifications, he helps customers adopt emerging technologies and build production-ready solutions on AWS.
Nitish Tipparaju is an Enterprise Support Manager at AWS, leading technical account teams that drive GenAI adoption and operational excellence for customers. He co-leads GenAI upskilling programs and brings hands-on experience helping organizations build their first AI agents on AWS.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article