Day 4 : Let’s Build a Knowledge-Based AI With Amazon Bedrock
I'll build a Knowledge Base using my notes stored in Amazon S3. Then I'll ask questions through the AWS console and check which sources the AI retrieves
1. Imagine this
It's the night before your exam. You have 80 pages of notes, and you need an answer to one question:
“What is a deadlock?”
Instead of searching through every page, imagine asking an AI and getting an answer based on your notes, along with references to the source material.
That's what we're going to build today.
2. What is a Knowledge Base? ( librarian example)
Imagine a librarian who has organized all your class notes.
When you ask a question, the librarian finds the most relevant pages and gives them to someone who can explain the answer.
Amazon Bedrock Knowledge Bases works in a similar way.
It helps retrieve relevant information from your documents and use that information to generate an answer.
This approach is called RAG (Retrieval-Augmented Generation).
The idea is simple:
Find relevant information first. Then generate an answer using that information.
3. Before you start
You'll need:
- An AWS account with access to Amazon Bedrock.
- A supported AWS Region and an available embedding and generation model.
- A source S3 bucket containing your notes from Day 3.
- Permission to create the required Knowledge Base resources.
- Around 20 minutes for a small practice setup, depending on resource creation and ingestion time.
Important: AWS features and model availability vary by Region. Check the current Amazon Bedrock documentation before starting.
4. Step 1 — Create a Knowledge Base
- Open the Amazon Bedrock console .
- Find Knowledge Bases in the navigation menu.
- Choose the option to create a Knowledge Base with a vector store.
- Give it a name, such as
my-class-notes. - Configure the required service permissions. You can use an automatically created service role if that option is available and appropriate.
- Select Amazon S3 as your data source and choose the bucket containing your notes.
Follow the setup wizard and review the configuration before creating the resource.
5. Step 2 — Choose an embedding model and vector store
For this example, I'll use Amazon Titan Text Embeddings V2, the embedding model I explored on Day 3 article.
The embedding model converts text into numerical vectors so the system can find semantically similar content.
Next, choose a vector store. For this tutorial, I'll use Amazon S3 Vectors, check if it's available in your Region and supported by the selected Knowledge Base configuration.
S3 Vectors stores and searches vector data without requiring you to provision a separate vector database cluster.
A cost warning before you continue
Not every vector store has the same pricing model.
Some options, including OpenSearch Serverless configurations, can incur ongoing capacity charges even when you're only experimenting. Actual costs depend on the service configuration and Region.
S3 Vectors is designed for cost-effective vector storage, but it still has storage and request charges. It isn't automatically free.
Before choosing, compare the current Amazon Bedrock pricing , Amazon S3 pricing , and the pricing for your selected vector store.
For a learning project, choose the simplest supported option that fits your budget.
6. Step 3 — Sync your notes
Creating a Knowledge Base doesn't automatically mean all your documents have been ingested.
You need to run the initial sync.
- Open your Knowledge Base.
- Find the data source you configured.
- Choose Sync or the equivalent ingestion action shown in your console.
- Wait for ingestion to finish successfully.
During ingestion, Amazon Bedrock processes your documents, splits them into chunks, generates embeddings, and indexes the resulting vectors.
With the default fixed-size chunking configuration, chunks are approximately 300 tokens with 20% overlap. The actual setup can differ if you choose another chunking strategy.
Whenever you add, modify, or remove source files, sync the data source again so those changes are reflected in the Knowledge Base.
7. Step 4 — Ask a question about your notes
Now comes the interesting part.
- Open your Knowledge Base in the Bedrock console.
- Open its testing interface.
- Select a supported generation model if prompted.
- Ask a question whose answer is present in your notes.
For example:
“What is a deadlock?”
Inspect the generated answer and the retrieved source references.
The source details can help you check which document sections were used to produce the answer.
However, a source reference doesn't guarantee that the answer is correct. Read the referenced text and compare it with the generated response.
8. Step 5 — Query the Knowledge Base using Python
After testing it in the console, we can use Python and Boto3 to query the same Knowledge Base.
Install Boto3 if you haven't already:
1
pip install boto3Here's a basic example using the
retrieve_and_generate() API:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import boto3
client = boto3.client(
"bedrock-agent-runtime",
region_name="YOUR_AWS_REGION"
)
response = client.retrieve_and_generate(
input={
"text": "What is a deadlock?"
},
retrieveAndGenerateConfiguration={
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": "YOUR_KB_ID",
"modelArn": "YOUR_MODEL_OR_INFERENCE_PROFILE_ARN"
}
}
)
print("Answer:")
print(response["output"]["text"])
print("\nSources:")
for citation in response.get("citations", []):
for reference in citation.get("retrievedReferences", []):
location = reference.get("location", {})
s3_location = location.get("s3Location", {})
uri = s3_location.get("uri")
if uri:
print(uri)
else:
print("Source location:", location)Replace these values
YOUR_AWS_REGION: The Region where your Knowledge Base is available.YOUR_KB_ID: The ID shown on your Knowledge Base details page.YOUR_MODEL_OR_INFERENCE_PROFILE_ARN: The supported generation model ARN or inference profile ARN accepted by your setup.
Make sure your AWS credentials are configured and your identity has the required permissions to invoke the API and access the Knowledge Base.
A common model error: Some models or configurations require an inference profile rather than a foundation-model ARN. Use the exact supported model or inference profile shown for your Region in the Bedrock console.
The
retrieve_and_generate() operation retrieves relevant information, generates a response, and returns source citations where applicable. You can read more in the Boto3 API documentation .9. Try this experiment
Let's test how well the Knowledge Base handles a question that isn't covered by your notes.
Ask:
“Who won the last Cricket World Cup?”
If your notes don't contain that information, check whether the system appropriately indicates that it couldn't find a relevant answer.
Don't assume it will always refuse to answer. The generation model may still use its general knowledge or produce an unsupported response.
This experiment helps you understand why retrieval quality, prompts, and checking sources matter.
10. Common mistakes and fixes
| Problem | Possible cause | What to try |
|---|---|---|
| No useful information is retrieved | Sync hasn't completed or the answer isn't in the documents | Check ingestion status and source files |
| New notes aren't found | Files were added after the last sync | Sync the data source again |
| Poor answers from a PDF | The file is scanned or its text isn't extracted correctly | Test with a text-based PDF or TXT file |
| AccessDeniedException | Missing permissions or model access | Check IAM permissions and model availability |
| Model ARN error | Unsupported model or required inference profile | Use a supported ARN for your Region |
| Unexpected AWS charges | A selected resource has ongoing charges | Review pricing and delete resources you no longer need |
11. Clean up your resources
Don't forget this step after experimenting.
- Delete the Knowledge Base when you no longer need it, following the console's deletion workflow.
- Check whether its associated vector store and other resources need separate cleanup.
- Delete the S3 Vectors bucket or index if it was created for this experiment and is no longer needed.
- Keep or delete your original notes bucket according to your needs.
- Review other resources created during setup and check AWS Billing for any ongoing charges.
Important: Deleting a Knowledge Base doesn't necessarily mean every related resource is automatically deleted. Check the console and the cleanup instructions for the resources you created.
12. Day 4 takeaway
Today, I learned how Amazon Bedrock Knowledge Bases connects my S3 notes to a retrieval-based AI workflow.
The process looks like this:
S3 notes → Ingestion and embeddings → Vector store → Relevant chunks → AI-generated answer with source references
I also learned that syncing is essential when documents change, model availability depends on the Region, and source references must be checked rather than blindly trusted.
The most important lesson for me: AI becomes more useful when it can retrieve relevant information from your own documents, but you still need to verify its answers.
Next: Day 5
Now that the Knowledge Base works, the next step is to build a simple chat interface so users can ask questions without opening the AWS console.
Now its Your turn: If you could ask your class notes one question and get the relevant source, what would you ask first? Share your idea in the comments . I am waiting for your comments !
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article