AWS Builder Center
Chunking Strategies for Amazon Bedrock Knowledge Bases

Chunking Strategies for Amazon Bedrock Knowledge Bases

Learn how chunking affects retrieval quality in Amazon Bedrock Knowledge Bases. Compare fixed-size, hierarchical, semantic, and no-chunking strategies, and understand how chunk size, overlap, structure, security, and cost influence RAG results.

Your RAG application can use a powerful model, a fast vector store, and carefully written prompts and still return the wrong answer.
The problem may have started much earlier, when the source documents were split into chunks.
Chunking determines what information becomes searchable as one unit. If a chunk is too large, retrieval may bring back unrelated details. If it is too small, the answer may be separated from the heading, condition, or warning that gives it meaning.

The real-world problem

Imagine a troubleshooting guide containing an error, its causes, permissions, resolution steps, and a warning. If the entire guide becomes one chunk, search may return unnecessary information. If every sentence becomes a chunk, the resolution may be separated from the error or warning.
Good chunking tries to preserve a complete, useful idea while keeping retrieval precise.

What is a chunk?

A chunk is a piece of source content that is processed, embedded, stored, and retrieved as a unit.
During ingestion, Bedrock parses the document, creates chunks and embeddings, and stores them with source information and metadata. At retrieval time, the selected chunks become context for the generation model.
The model can use only the evidence that survives this process. Prompt engineering cannot repair a poorly formed chunk.

The main chunking strategies

The available options depend on the knowledge-base type and configuration. Customer-managed vector knowledge bases provide the broader set of strategies described below. The fully managed Bedrock Knowledge Base experience currently offers built-in/default or fixed-size chunking.

Default chunking

When no custom configuration is provided for a customer-managed knowledge base, Amazon Bedrock splits text into chunks of approximately 300 tokens while preserving sentence boundaries.
This is a practical starting point for simple prose but may struggle with long procedures, complex tables, or strongly defined sections.

Fixed-size chunking

Fixed-size chunking divides content using a maximum number of tokens and an overlap percentage.
It is predictable and easy to evaluate, but a fixed boundary may split a logical section.
Important settings:
  • Maximum tokens controls the upper size of each chunk.
  • Overlap repeats part of one chunk in the next so information near a boundary is not lost.
More overlap preserves continuity but creates duplicate text, raises costs, and may produce repetitive results.

Hierarchical chunking

Hierarchical chunking creates smaller child chunks and larger parent chunks.
Child chunks support precise retrieval. When one matches, Bedrock can replace it with the broader parent so the model receives more context.
It can return fewer final results than requested when matching children share a parent.

Semantic chunking

Semantic chunking finds changes in meaning instead of splitting at fixed token counts. Related sentences stay together while topic shifts create boundaries.
Its configuration includes:
  • a maximum number of tokens;
  • a buffer of surrounding sentences used when evaluating boundaries;
  • a breakpoint threshold controlling how different adjacent content must be before it is split.
It can create coherent units but uses a foundation model during ingestion and adds cost.

No chunking

With no chunking, each source document is treated as one chunk.
This works when files are already small and focused. For long documents, it usually produces overly broad retrieval units.

How chunk size changes retrieval

Chunk size creates a precision-versus-context trade-off.

Smaller chunks

Smaller chunks match narrow questions precisely but may lose nearby definitions, headings, exceptions, or steps.

Larger chunks

Larger chunks preserve context but may represent several topics, reducing precision and increasing generation tokens.
There is no universal best size. The right choice depends on the documents, user questions, model limits, and retrieval settings.

Follow the document structure

Good boundaries often occur at headings, paragraph groups, procedure steps, tables, question-and-answer pairs, and policy clauses. A structure-aware parser or custom transformation helps preserve these relationships. Table rows should keep their headers, and warnings should stay with the steps they modify.

A practical evaluation process

Test direct factual questions, questions needing nearby context, exact error codes, boundary cases, long procedures, and questions that should return no answer.
For every query, check whether the correct chunk was retrieved, it contained enough context, irrelevant content was excluded, citations were accurate, and the answer remained faithful.
Compare retrieval relevance, answer faithfulness, latency, and cost across representative questions.

Important design trade-offs

  • Precision versus context: Small chunks retrieve narrowly; large chunks preserve more surrounding meaning.
  • Overlap versus duplication: Overlap protects boundary information but increases storage and repeated results.
  • Quality versus ingestion cost: Semantic chunking may create better boundaries but requires additional model processing.
  • Standardization versus document awareness: One configuration is easier, but different document types may need different treatment.

Security and cost considerations

Chunk boundaries affect security. A chunk must not mix content from different tenants or permission levels.
  • Attach authorization metadata to every chunk.
  • Apply entitlement filters during retrieval.
  • Avoid mixing sensitive and public material.
  • Remove deleted or outdated content from the index.
Smaller chunks create more vectors; large chunks send more text to the model. Both can increase cost.

Common mistakes: what to check and how to fix them

Common mistake
What to check
How to fix it
Retrieved text is relevant but incomplete
Check whether headings, conditions, or warnings fall into adjacent chunks
Increase chunk size or overlap, or use hierarchical chunking
Results contain several unrelated topics
Inspect the source span covered by each chunk
Reduce chunk size or use semantic or structure-aware boundaries
The same passage appears repeatedly
Review overlap and near-duplicate retrieval results
Reduce overlap and consider deduplication or reranking
Tables lose their meaning
Confirm whether headers and rows remain connected after parsing
Use a parser or custom transformation that preserves table structure
Exact answers are not retrieved
Test the query against individual chunk text and metadata
Preserve key terms, improve boundaries, and consider hybrid search
A new strategy does not appear after editing the data source
Check the original data-source chunking configuration
Create a new data source and re-ingest; the strategy cannot be changed after connection
Costs rise sharply after re-indexing
Compare chunk count, semantic-processing usage, and vector dimensions
Use fewer meaningful chunks, reduce unnecessary overlap, or choose a simpler strategy
Users retrieve content they should not see
Inspect metadata and filters on every returned chunk
Apply tenant and entitlement metadata during ingestion and enforce it during retrieval

Key takeaways

Chunking controls the evidence your RAG application can retrieve. Fixed-size chunking is predictable, hierarchical chunking balances precise search with broader context, semantic chunking follows meaning, and no chunking works only when documents are already small and focused.
Start with a simple strategy, test it using real questions, and inspect the retrieved chunks not only the final answers.
The best chunk is not the smallest or largest. It is the smallest unit that still contains everything needed to understand and support the answer.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article