Recordings Become Retrievable Knowledge: Bedrock Data Automation into Knowledge Bases
Your largest corpus is the one nobody indexed. Here's how to turn audio and video into grounding data an agent can cite — and which of the three ingestion paths to actually pick.
Every organization is sitting on years of recorded meetings, support calls, training sessions, product demos, and screen recordings. Almost none of it is retrievable. When someone asks "what did we decide about the vendor migration," the answer exists — in a 47-minute recording nobody will ever scrub through.
The reflex fix is to bolt transcription onto your RAG pipeline: run an ASR service, dump the text into S3, index it. It works, sort of, and then you discover what you lost. The transcript has no timestamps you can link back to, no slide content, no distinction between the presenter reading a bullet and someone in the room disagreeing with it. Your agent can now quote the meeting but can't tell you when it happened.
Amazon Bedrock Knowledge Bases closes that gap. It supports multimodal content including images, audio, and video, letting you search product catalogs using images, find specific moments in training videos, and retrieve relevant segments from support call recordings.
The catch is that there are now three ways to get media into a knowledge base, they behave very differently, and one of them has a documented limitation that will silently break the most common use case. That's what this article is about.
This is a companion to my earlier piece on connecting a Bedrock Managed Knowledge Base to LangGraph over MCP. That one covered retrieval. This one covers what you feed it.
Architecture
Figure 1 — Three ingestion paths converging on one retrieval layer. The path you choose determines whether your agent can search what people said, what they showed, or both.
The three paths, stated plainly
Path A — Managed Knowledge Base with Smart Parsing. The managed offering uses Smart Parsing to automatically determine the optimal parsing strategy for each data type, preserving metadata, embedded images, and thread context while handling text, images, audio, video, and multiple languages — all without manual configuration. Zero parser config. Least control.
Path B — Vector-store knowledge base with Bedrock Data Automation as the parser. BDA converts multimedia to text representations: audio transcribed via ASR, video processed to extract scene summaries and transcripts, images through OCR and visual content extraction. You can use BDA standalone or as a parser in Knowledge Bases RAG workflows.
Path C — Nova Multimodal Embeddings. Preserves native format for visual and audio similarity matching, embedding images, audio, and video directly without conversion to text.
Path C sounds like the most modern option, and for image search it is. But read this before you build on it:
Nova Multimodal Embeddings v1.0 has limited support for searching speech content in audio and video data. If you need to support speech, use Bedrock Data Automation as a parser.
That is AWS's own guidance, and it is the single most important sentence in this article. "Find the moment in the meeting where we decided X" is a speech query. If you pick the native-embedding path because it sounds better on paper, semantic search over your meeting corpus will underperform in a way that looks like bad retrieval tuning rather than a wrong architectural choice.
So: if your corpus is recorded speech, use BDA. If it's visual similarity — product photos, screenshots, diagrams — use Nova. If it's both, that's two data sources, not one clever configuration.
Prerequisites
- Permissions to create S3 buckets, Amazon Bedrock resources, vector store resources, and the required service roles.
- Two S3 buckets for the vector-store path: one for source documents, one for intermediate files generated during multimodal processing. The knowledge base manages objects in the second bucket — you create it and then leave it alone.
- A region where your chosen approach is available. Multimodal processing approaches have different regional availability, so confirm before you design around one.
1
pip install boto3
Step 1 — Choose the parser, knowing what each one ignores
When you create a knowledge base with a vector store, you pick a parser. There are three, and the default one will quietly skip your media:
| Bedrock default parser | Bedrock Data Automation | Foundation model parser | |
|---|---|---|---|
| Handles | Word, Excel, HTML, Markdown, TXT, CSV | PDFs, images, audio, video | PDFs, images, structured docs, tables |
| Media behaviour | Ignores multimodal files | Transcribes audio, summarizes video scenes, OCRs images | Extracts text, describes visual elements |
| Audio output | — | ASR transcript | — |
| Video output | — | Scene summaries + transcript + on-screen text | — |
| Best for | Text-only corpora | Recordings, media libraries | Visually rich documents, complex layouts |
| The trap | Silently drops your .mp4 | Async — results land in S3, not inline | Not built for audio or video |
The first column is where people lose a week. The default parser processes common text formats while ignoring multimodal files — it does not error, it does not warn, it just doesn't index them. Your sync completes successfully, your knowledge base reports documents ingested, and every question about a recording returns nothing relevant.
If you take one operational habit from this article: after your first sync, query for something you know is only spoken aloud in a video. If it comes back empty, your parser is wrong.
Step 2 — Run Bedrock Data Automation
BDA supports four modalities — documents, images, video, and audio — and when used in an application all modalities go through the same asynchronous inference API with results written to an S3 bucket. One API, four content types, output in S3.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import boto3
bda = boto3.client("bedrock-data-automation-runtime", region_name="us-west-2")
response = bda.invoke_data_automation_async(
inputConfiguration={
"s3Uri": "s3://meetings-raw/standup-2026-08-14.mp4"
},
outputConfiguration={
"s3Uri": "s3://meetings-processed/"
},
dataAutomationConfiguration={
"dataAutomationProjectArn": PROJECT_ARN,
"stage": "LIVE",
},
)
invocation_arn = response["invocationArn"]
The async shape is not an inconvenience to work around — it's the correct model for a 47-minute video, and it means your ingestion is a job queue rather than a request/response path. Build for that: track invocation ARNs, poll or subscribe to completion, and make the whole thing idempotent so a re-run doesn't duplicate chunks.
For video, the results include a summarization of the overall video, a summary scene by scene, and the text that appears during the video. That third one matters more than it sounds. On-screen text is where the architecture diagram's labels live, where the slide's bullet points live, and where the numbers someone read out incorrectly actually appear correctly.
BDA also ships built-in safeguards including visual grounding and confidence scores, which are what let you set a quality floor at ingestion rather than discovering a bad transcription through a wrong answer three weeks later.
Step 3 — Configure chunking, in the place it actually lives
This is the configuration mistake that produces the most confusing error message in the whole pipeline.
Audio and video chunking parameters are configured at the embedding model level, not at the data source level. A validation exception occurs if you provide this configuration for a non-multimodal embedding model.
So the sequence matters: pick the embedding model first, then configure chunk duration. Do it the other way around and you get a validation exception that names the configuration rather than the actual cause, and you will spend an afternoon reading data source docs that have nothing to do with it.
The knob itself is straightforward: audio and video chunk duration defaults to 5 seconds with a range of 1 to 30 seconds, where shorter chunks enable precise content retrieval and longer chunks preserve more semantic context.
That trade-off deserves a real decision rather than accepting the default:
| Chunk duration | Retrieval behaviour | Fits |
|---|---|---|
| 1–3s | Pinpoint moments; fragments lack context | Compliance review, exact-quote lookup |
| 5s (default) | Balanced; a sentence or two per chunk | Most meeting and call corpora |
| 10–15s | Whole thoughts survive; less precise seeking | Lectures, training, long-form explanation |
| 20–30s | Strong topical context; weak pinpointing | Podcasts, panels, discovery-style search |
The instinct is to go short for precision. Resist it for conversational audio: people take longer than three seconds to express a complete thought, and a 2-second chunk of someone mid-sentence embeds badly and retrieves worse.
Step 4 — Pick the embedding model deliberately
Two viable combinations, and they are not interchangeable:
BDA + a text embedding model (such as Titan Text Embeddings v2). BDA converts everything to text; the text embedding model indexes it. Text embedding models limit retrieval to text-only content, but you enable multimodal retrieval by selecting either Bedrock Data Automation or a foundation model as the parser. This is the path for speech.
Nova Multimodal Embeddings v1.0. Choose it for direct visual and audio similarity searches — images, audio, and video embedded natively without conversion to text. This is the path for "find me more images like this one," and it supports image-based querying, which the text path cannot do at all.
The honest summary: Nova is better at similarity, BDA is better at speech. Most enterprise recording corpora are a speech problem wearing a video costume.
Step 5 — Keep the original media reachable
Optional in the docs, non-negotiable in practice.
If configured, original multimedia files are copied to a dedicated multimodal storage destination for reliable retrieval, ensuring availability even if source files move. Embeddings are stored alongside metadata including file references, timestamps for audio and video, and content type.
Configure it. Here's why: the entire value of media RAG is that a citation resolves to a moment, not a filename. That requires two things to survive ingestion — the timestamp, which Knowledge Bases stores for you, and the media file itself, which it only stores if you ask. If someone reorganizes the source bucket six months from now, every citation in your application breaks unless you enabled multimodal storage.
What happens at query time
Figure 2 — Ingestion runs once per recording; retrieval runs per question. The timestamp survives both, which is the point.
Retrieval is unchanged by the fact that the source was a video, which is the entire payoff. On the managed knowledge base, Agentic Retrieval understands user intent, identifies the most relevant data sources, and iterates through multiple retrieval steps in a single API call with no custom orchestration code. Your recordings are now just documents that happen to have timestamps.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
agent_rt = boto3.client("bedrock-agent-runtime", region_name="us-west-2")
resp = agent_rt.retrieve(
knowledgeBaseId=KB_ID,
retrievalQuery={"text": "What did we decide about the vendor migration?"},
retrievalConfiguration={
"vectorSearchConfiguration": {"numberOfResults": 8}
},
)
for r in resp["retrievalResults"]:
loc = r.get("location", {})
print(round(r.get("score", 0), 3),
loc.get("s3Location", {}).get("uri"),
r.get("metadata", {}).get("startTime"))
Retrieved results include source attribution for visual data, which is what turns "the deck said Q3" into a citation someone can open and check.
Making citations survive the pipeline
This is where most media-RAG implementations quietly fail. The transcript is indexed, retrieval works, the answer is correct — and the citation says
standup-2026-08-14.mp4, which is useless to someone who wants the twelve seconds where the decision was made.Three rules:
- Read the timestamp metadata and carry it through your application. Knowledge Bases stores it. If your API response drops it on the way to the UI, you have a filename-only citation with extra steps.
- Construct a deterministic deep link. A presigned URL to the multimodal storage copy plus a
#t=fragment turns every citation into a playable link. Build it once at response time from the metadata you already have. - Show the modality. A user reads a quote from a transcript differently when they know it was spoken versus written on a slide. Content type is in the metadata; surface it.
The test: ask your agent a question, take the citation it returns, and try to reach the exact moment in the recording. If you can't do it in one click, the pipeline isn't finished, however good the answer text looks.
Troubleshooting
| Symptom | Likely cause | What to do |
|---|---|---|
| Sync succeeds, media never retrievable | Bedrock default parser ignores multimodal files | Switch the data source to BDA or a foundation model parser and re-sync |
| Validation exception on chunking config | Chunk duration set for a non-multimodal embedding model | Configure audio/video chunk duration at the embedding model level, after choosing a multimodal-capable setup |
| Speech queries return poor results | Nova Multimodal Embeddings used for a speech corpus | Re-ingest with BDA as parser plus a text embedding model |
| Image-based queries unsupported | Text embedding model in use | Use Nova Multimodal Embeddings for image-as-query search |
| BDA call returns immediately with no output | Async by design; results land in S3 | Track the invocation ARN and read the output bucket on completion |
| Fragmented, incoherent retrieved chunks | Chunk duration too short for conversational speech | Raise from the 5s default toward 10–15s and re-evaluate |
| Citations resolve to a file, not a moment | Timestamp metadata dropped in the application layer | Read timestamps from results and build deep links |
| Citations break after a bucket reorganization | Multimodal storage destination not configured | Enable it so originals are copied for reliable retrieval |
| Feature missing in your region | Multimodal approaches vary by region | Check availability for your specific approach before designing |
Where to go next
The natural follow-on is a routing layer at ingestion: send call recordings down the BDA-plus-text-embeddings path, send screenshot and product-image libraries down the Nova path, and keep them as separate data sources on the same knowledge base. You get image-as-query search where it helps and reliable speech search where it matters, instead of one compromise configuration that half-serves both.
The second thing worth building is an ingestion quality gate. BDA returns confidence scores; a recording whose transcript scores poorly should be flagged before it becomes a confidently wrong citation, not after.
References
- Build a knowledge base for multimodal content — how BDA and Nova differ, and what metadata gets stored alongside embeddings.
- Create a knowledge base for multimodal content — parser options, the embedding-model-level chunking rule, and the speech limitation on Nova.
- Amazon Bedrock Knowledge Bases — Smart Parsing and Agentic Retrieval on the managed offering.
- Get insights from multimodal content with Amazon Bedrock Data Automation — standalone vs. parser usage.
- New Amazon Bedrock capabilities enhance data processing and retrieval — the four modalities, async API, visual grounding and confidence scores, and what video output contains.
- Amazon Bedrock Data Automation is now generally available
- Amazon Bedrock Knowledge Bases now processes multimodal data — source attribution for visual data.
- Building a multimodal RAG application using Bedrock Data Automation and Bedrock Knowledge Bases — an end-to-end reference build.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article