
The DynamoDB Vector Index Is Live: Building a RAG Agent Memory Store with Amazon Bedrock and Titan Embeddings
A hands on tutorial where we generate embeddings with Titan on Bedrock, write them with PutItem, create a DynamoDB vector index, query it with SearchVectors, and wire the whole thing into a Bedrock Agent so the agent ends up with real long term memory instead of a context window that resets every session.
If you have built a Retrieval Augmented Generation pipeline, or an agent that is supposed to "remember" things, you have probably run into this exact pattern. Your operational data sits in DynamoDB because that is where your application already lives. Your embeddings sit somewhere else, maybe OpenSearch, maybe Pinecone, maybe a self hosted FAISS index someone spun up during a hackathon and never tore down. A change to a record means updating two systems instead of one. A single search means two network round trips, one to the vector store to get back a list of IDs, and a second one back to DynamoDB to actually fetch the item data those IDs point to. Two databases to operate, patch, and pay for, when conceptually you only needed one.
In August 2026 AWS shipped native vector search for DynamoDB, and it is now generally available. It is a new vector index type that lives directly on your existing table. You store an embedding as a plain list of numbers on an item, right next to its normal attributes, and DynamoDB indexes that list for approximate nearest neighbor search. One table holds the data. One write updates both the record and the index. One query returns the similarity score and the full item together, with no second lookup needed.
This tutorial turns that capability into something concrete and genuinely useful, an agent memory store. We are going to give a Bedrock Agent the ability to remember facts, decisions, and conversation summaries as it works, and to recall the most relevant ones semantically rather than through an exact key lookup that only works if you already know exactly what you are looking for.
By the time you are done reading and following along, you will have
- A DynamoDB table with a vector index sized correctly for Titan Text Embeddings V2
- A working embed, write, and search loop in Python that you can run yourself
- A retrieve_memory tool wired into a Bedrock Agent, with notes on how the same code drops into LangChain
- A working mental model of the moving parts, the tradeoffs you are making, and the mistakes people tend to hit the first time
Two flows, one table: writing memory and recalling memory
At a high level there are two flows running through this system. One is writing memory, where the agent produces a piece of text worth remembering, embeds it, and stores it. The other is recalling memory, where a new query gets embedded and used to search everything stored so far. Both flows pass through Bedrock to turn text into vectors, and both land on the same DynamoDB table for storage and search.

The architectural point worth sitting with here is that the vector index is just another index on the same table, similar in spirit to a global secondary index. There is no second system that needs to stay in sync, and there is no second network hop required to fetch the underlying item once you have a match. SearchVectors hands back the item's projected attributes directly as part of the response.
Four pieces, one clean setup: what you are assembling
Before jumping into code it helps to know the shape of the four pieces you are about to assemble and how they relate to each other.
Component | Role | Notes |
|---|---|---|
Embedding model, Bedrock Titan Text Embeddings V2 | Turns text into a fixed length float vector | Output dimension, say 1024, has to match the vector index's configured Dimensions exactly, or writes and searches will not line up |
Vector attribute | A regular DynamoDB item attribute, stored as a List of Numbers | There is no new data type here, it rides on the existing DynamoDB List type you already know |
Vector index | A new secondary index type, created through CreateTable or UpdateTable with a VectorIndexes parameter | Defines dimensions, distance function, projection, and optionally a partition key plus inline filters |
SearchVectors API | Runs approximate nearest neighbor search against the vector index | Returns ranked items along with a similarity Score, though the raw vector itself is not returned by default |
One distinction worth locking in early, a vector index is not a GSI and it is not an LSI. Query and Scan still do exact match and range lookups the way they always have. The vector index answers a different kind of question entirely, something closer to "what is nearest to this vector," and it only answers that question through SearchVectors.
Step 1, standing up the AgentMemory table and its vector index
For an agent memory store, each item represents one "memory," which could be a fact, a decision, or a compact conversation summary, and it should be scoped to a session or a user so one agent's memories do not bleed into another's.

A few design decisions are worth calling out explicitly here, because they are easy to get wrong on a first pass.
- The partition key on the base table, memory_id in this case, is completely independent from the vector index's own optional partition key. For an agent handling many concurrent sessions, scoping the vector index itself by session_id means every search only touches one session's memories. That makes each search both faster and cheaper, since DynamoDB has far less data to sift through.
- memory_type and session_id both make good inline filter candidates. They let you say "only search this session's memories" or "only look at facts, not summaries" without running a second query afterward to filter results down.
- Keep the embedded text short and focused, ideally a single fact or a compact summary rather than a full transcript. Embedding quality tends to degrade on long, unfocused text, and DynamoDB's per item size limit still applies to the whole item, vector included, so there is a practical ceiling here as well as a quality one.
Create the table with a vector index
import boto3
dynamodb = boto3.client("dynamodb")
dynamodb.create_table( TableName="AgentMemory", AttributeDefinitions=[ {"AttributeName": "memory_id", "AttributeType": "S"}, ], KeySchema=[ {"AttributeName": "memory_id", "KeyType": "HASH"}, ], VectorIndexes=[ { "IndexName": "MemoryIndex", "VectorAttribute": {"AttributeName": "embedding"}, "Dimensions": 1024, # Must match Titan V2 output "DistanceFunction": "DOT_PRODUCT", # Titan embeddings are normalized "Projection": {"ProjectionType": "ALL"}, "SearchSchema": { # Every search must scope to one session_id, which keeps # searches fast and cheap as memory grows across many sessions. "VectorIndexPartitionKey": {"AttributeName": "session_id"}, "InlineFilterAttributes": [ {"AttributeName": "memory_type"}, ], }, } ], BillingMode="PAY_PER_REQUEST", # Vector indexes require on-demand capacity)print("Creating AgentMemory table with MemoryIndex. This happens asynchronously.")
A handful of things trip people up the first time they run this.
- Dimensions has to match your embedding model's output exactly. Titan Text Embeddings V2 can emit 256, 512, or 1024 dimensions depending on how you call it, so pick one value and configure both the model call and the index the same way, every time.
- Billing only works in on demand mode right now. Vector indexes require PAY_PER_REQUEST, so you cannot attach one to a table running in provisioned capacity mode.
- Index configuration is immutable once set. Dimensions, distance function, projection, and the partition key cannot be changed after creation. If you need to change any of these later, you create a brand new index instead, and DynamoDB backfills it for free from the vector attribute you already have stored, so this is more of an inconvenience than a real blocker.
- Table and index creation both happen asynchronously. Poll DescribeTable and check that VectorIndexes has IndexStatus equal to ACTIVE and Backfilling equal to False before you start writing data, otherwise early writes may not get indexed the way you expect.
Step 2, giving the agent something to remember
import jsonimport timeimport ulid # pip install python-ulid
bedrock = boto3.client("bedrock-runtime")
def embed(text: str) -> list[float]: """Generate a 1024-dim embedding with Titan Text Embeddings V2.""" response = bedrock.invoke_model( modelId="amazon.titan-embed-text-v2:0", contentType="application/json", accept="application/json", body=json.dumps({ "inputText": text[:8000], # stay under Titan's input limit "dimensions": 1024, # must match the vector index "normalize": True, # lets us use DOT_PRODUCT as cosine }), ) return json.loads(response["body"].read())["embedding"]
def remember(session_id: str, text: str, memory_type: str = "fact") -> str: """Embed a piece of agent memory and store it in DynamoDB.""" memory_id = str(ulid.new()) vector = embed(text)
dynamodb.put_item( TableName="AgentMemory", Item={ "memory_id": {"S": memory_id}, "session_id": {"S": session_id}, "memory_type": {"S": memory_type}, "text": {"S": text}, "embedding": {"L": [{"N": str(v)} for v in vector]}, "created_at": {"S": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())}, }, ) return memory_id
# Example, the agent learns something during a conversationremember( session_id="sess-7f3a", text="The customer's production DynamoDB table is named 'orders-prod' and lives in us-east-1.", memory_type="fact",)remember( session_id="sess-7f3a", text="Decided to recommend a partition-key-scoped vector index because the customer expects 50M+ memories.", memory_type="decision",)
There is nothing exotic happening here. It is a normal PutItem call, full stop. The embedding attribute is a List of Numbers, exactly like any other list attribute you might already have on an item, since DynamoDB does not introduce a new data type just for this.
Step 3, asking the index what it remembers
def recall(session_id: str, query: str, memory_type: str | None = None, top_k: int = 5): """Retrieve the most relevant memories for a session, ranked by similarity.""" query_vector = embed(query)
params = { "TableName": "AgentMemory", "IndexName": "MemoryIndex", "SearchVector": [{"N": str(v)} for v in query_vector], "TopK": top_k, # Required because the index defines a vector-index partition key "SearchConditionExpression": "session_id = :sid", "ExpressionAttributeValues": {":sid": {"S": session_id}}, "ReturnConsumedCapacity": "INDEXES", } if memory_type: params["SearchConditionExpression"] += " AND memory_type = :mt" params["ExpressionAttributeValues"][":mt"] = {"S": memory_type}
response = dynamodb.search_vectors(**params) return [ { "text": r["Item"]["text"]["S"], "type": r["Item"]["memory_type"]["S"], "score": r["Score"], } for r in response["SearchResults"] ]
for m in recall("sess-7f3a", "what database is the customer using in production?"): print(f"[{m['score']:.4f}] ({m['type']}) {m['text']}")
Two behaviors are worth calling out here specifically because they differ from what you are used to with Query or Scan.
- Eventual consistency applies. The vector index updates asynchronously after a write, so a memory written a moment ago might not show up in search results immediately. For an agent that just learned something and needs to recall it within the very same turn, keep that freshest piece of context in the prompt directly rather than counting on the index to have already caught up.
- The vector itself is not returned by default. What you get back in SearchResults is the projected attributes, things like text, type, and timestamp, not the raw embedding. That also keeps the bytes processed, and therefore the cost of the query, lower than it would otherwise be.
Step 4, giving the agent the ability to call recall itself
Here is the part that actually makes this useful, turning recall into a tool an LLM agent can call on its own when it decides it needs more context. This works the same way whether you are using a Bedrock Agent action group or a LangChain or LangGraph tool. The retrieval logic underneath is identical either way, only the tool registration wrapper around it changes.

A minimal Bedrock Agent action group Lambda handler looks like this.
def lambda_handler(event, context): """Bedrock Agent action-group handler for the retrieve_memory tool.""" params = {p["name"]: p["value"] for p in event["parameters"]} session_id = params["session_id"] query = params["query"]
memories = recall(session_id, query, top_k=5) context_text = "\n".join(f"- ({m['type']}) {m['text']}" for m in memories)
return { "response": { "actionGroup": event["actionGroup"], "function": event["function"], "functionResponse": { "responseBody": {"TEXT": {"body": context_text or "No relevant memory found."}} }, } }
If you would rather stay inside LangChain, the same recall function drops straight into a tool decorated function and binds to any AgentExecutor or LangGraph node without modification. The DynamoDB and Bedrock plumbing underneath does not change at all, it is only the outer wrapper that differs depending on which framework you are standardizing on.
A LangChain version of the same tool looks close to this.
from langchain_core.tools import tool
@tooldef retrieve_memory(session_id: str, query: str, memory_type: str | None = None) -> str: """Look up relevant past facts, decisions, or summaries for this session.""" memories = recall(session_id, query, memory_type=memory_type, top_k=5) if not memories: return "No relevant memory found." return "\n".join(f"- ({m['type']}) {m['text']}" for m in memories)
Once that tool is bound to an agent, the model itself decides when a question needs a memory lookup and when it can answer directly from the current conversation. You do not need to hardcode "always check memory first," the agent figures that out from the tool description and the shape of the question it is facing.
Deciding what actually gets written to memory
A question that comes up almost immediately once this is running, what should the agent actually remember. Writing every single message to the memory table is technically possible, but it tends to produce a lot of low value noise that makes recall worse rather than better, since the index ends up full of small talk and half formed thoughts sitting next to the facts that actually matter.
A more workable pattern is to have the agent, or a lightweight summarization step running alongside it, decide what is worth keeping. This usually falls into one of three buckets.
- Facts, which are stable pieces of information the agent picked up along the way, things like a customer's account configuration, a preference they stated, or a constraint they mentioned. These tend to stay valid for a long time and are worth storing as their own small item.
- Decisions, which capture something the agent chose to do and, ideally, the reasoning behind the choice. These are useful later both for the agent itself and for a human reviewing what the agent did and why.
- Summaries, which compress a longer stretch of conversation into a few sentences. These are useful when you want the gist of an exchange without keeping every message that led up to it.
A simple way to generate these is to run a short summarization prompt against the foundation model at the end of a session, or after some number of turns, and have it emit a small list of candidate memories in a structured format your application can parse directly into calls to remember.
def summarize_session_for_memory(transcript: str) -> list[dict]: """Ask the model to extract durable facts, decisions, and summaries.""" response = bedrock.invoke_model( modelId="anthropic.claude-sonnet-5", contentType="application/json", accept="application/json", body=json.dumps({ "anthropic_version": "bedrock-2023-05-31", "max_tokens": 500, "messages": [{ "role": "user", "content": ( "Read this conversation transcript and extract durable memories worth " "keeping. Return a JSON list of objects with 'text' and 'memory_type' " "(one of fact, decision, summary). Only include things worth remembering " "beyond this session.\n\n" + transcript ), }], }), ) result = json.loads(response["body"].read()) return json.loads(result["content"][0]["text"])
def close_out_session(session_id: str, transcript: str): """Extract and store memories at the end of a session.""" for memory in summarize_session_for_memory(transcript): remember(session_id, memory["text"], memory_type=memory["memory_type"])
This keeps the memory table lean and keeps recall quality high, since every item in the index is something a summarization pass judged worth keeping, rather than raw conversational exhaust.
Handling throttling and partial failures gracefully
Any code calling two AWS services back to back, Bedrock for embeddings and DynamoDB for storage or search, needs to handle transient failures without falling over. A throttling exception from either service should not take down the whole agent turn, and a slow embedding call should not hang the conversation indefinitely.
A basic retry wrapper around the embed function covers most of what you need for a first production pass.
import randomfrom botocore.exceptions import ClientError
def embed_with_retry(text: str, max_attempts: int = 4) -> list[float]: """Wrap embed() with exponential backoff for throttling and transient errors.""" for attempt in range(max_attempts): try: return embed(text) except ClientError as e: code = e.response["Error"]["Code"] if code in ("ThrottlingException", "ServiceUnavailableException") and attempt < max_attempts - 1: delay = (2 ** attempt) + random.uniform(0, 0.5) time.sleep(delay) continue raise raise RuntimeError("Exceeded retry attempts for embedding call")
The same pattern applies to the DynamoDB calls themselves. Boto3 already retries throttling on DynamoDB requests by default, so you generally do not need to write your own retry loop for put_item or search_vectors calls, but it is worth setting an explicit retry configuration on the client rather than relying on the SDK defaults if this table is going to see meaningful production traffic.
from botocore.config import Config
dynamodb = boto3.client( "dynamodb", config=Config(retries={"max_attempts": 5, "mode": "adaptive"}),)
For the agent itself, a reasonable failure mode when recall fails outright is to let the agent answer from its existing context and note that it could not check its memory, rather than surfacing a raw exception back to the user or silently returning no results and letting the model guess. Wrapping the Lambda handler in a try block that returns a clear, model readable error message accomplishes this without much extra code.
Checking that recall is actually working
Once the write and search loop is running, it is worth spending some time confirming that recall behaves the way you expect before wiring it into a live agent. A vector index can be technically working, meaning it returns results without erroring, while still returning results that are not actually useful, and that distinction only shows up if you go looking for it.
A simple evaluation approach that works well here is building a small set of query and expected memory pairs by hand, running each query through recall, and checking whether the expected memory shows up in the top few results.
EVAL_CASES = [ { "session_id": "sess-7f3a", "query": "what database is the customer using", "expected_substring": "orders-prod", }, { "session_id": "sess-7f3a", "query": "why did we scope the vector index by partition key", "expected_substring": "50M+ memories", },]
def run_recall_eval(): hits = 0 for case in EVAL_CASES: results = recall(case["session_id"], case["query"], top_k=3) found = any(case["expected_substring"] in r["text"] for r in results) hits += int(found) status = "PASS" if found else "FAIL" print(f"[{status}] {case['query']}") print(f"\n{hits}/{len(EVAL_CASES)} passed")
run_recall_eval()
This is deliberately simple, nothing more than string matching against known answers, but it catches the two most common failure modes early. Either the embedding model is not capturing the meaning of what was stored well enough for the query to find it, or the partition key and filter scoping is excluding items it should be including. Both are far easier to fix before an agent is relying on this in front of real users than after.
As the memory table grows past a few hundred items, it is worth expanding this into a slightly more thorough evaluation set that covers paraphrased queries, queries using different wording than the original stored text, and queries that should return nothing relevant at all, so you can confirm the system handles a true negative gracefully rather than forcing a match on the nearest thing it can find.
How this compares to running OpenSearch or a dedicated vector database
DynamoDB's vector index is not a wholesale replacement for every vector search workload, and it helps to be clear eyed about where it fits best and where a dedicated vector database still makes more sense.
Situation | DynamoDB vector index | Dedicated vector database |
|---|---|---|
Your data already lives in DynamoDB as the system of record | Strong fit, no synchronization pipeline needed | Adds a second data layer and a sync pipeline you now have to maintain |
You need advanced filtering, ranges, or hybrid keyword plus vector search | Limited, inline filters only support equality | Often stronger, purpose built query languages support this natively |
Your workload is pure vector search with no operational data attached | Workable, but you gain little from DynamoDB's operational features | Often a better fit, since that is what these systems are optimized for |
You want a single serverless bill and no infrastructure to manage | Strong fit, same pay per request model as the rest of DynamoDB | Varies, many vector databases still require cluster sizing and management |
You need to scale to very large vector counts with predictable latency | Strong fit, designed for this from the ground up | Also strong, though often requires more manual capacity planning |
For an agent memory store specifically, the calculus tends to favor DynamoDB pretty clearly, since the memories themselves are operational data belonging to a session or a user, the filtering needs are usually simple equality checks like session or memory type, and the whole point is avoiding a second system to keep in sync. A pure document search engine indexing millions of unrelated documents with rich faceted filtering is a different problem, and that is where a dedicated vector database or OpenSearch is still often the better call.
Tuning the index for how your agent actually searches
Decision | Choose this when | Watch out for |
|---|---|---|
Distance function | DOT_PRODUCT when embeddings are normalized, like Titan with normalize set to True. COSINE when they are not normalized. EUCLIDEAN when magnitude itself carries meaning | Mixing a normalized model with EUCLIDEAN, or forgetting to normalize and reaching for DOT_PRODUCT anyway, silently skews your rankings without throwing any error |
Vector index partition key | High memory volume, meaning many sessions or many users, or generally high search throughput | Every search must supply this value once it is set, an index with a partition key cannot be searched "globally" across everything |
Inline filter attributes | You need to scope results by category, type, or status alongside similarity | Only exact match equality is supported, there are no ranges, no IN clauses, and no prefix filters |
Projection, ALL versus INCLUDE | ALL for simplicity when items are small. INCLUDE to project only the specific fields you actually need | A larger projection increases both storage cost and the bytes processed on every single search |
Embedding granularity | One fact, decision, or summary per item, never a whole raw transcript | Long, unfocused text produces vaguer embeddings and noticeably weaker recall quality |
What to watch for the first time you build this
- Forgetting the partition key on search. If the index defines a VectorIndexPartitionKey, a SearchVectors call without a matching SearchConditionExpression fails outright. This is enforced by the service, not something you can skip if you forget.
- Dimension mismatch. Switching embedding models, or even bumping a model version, without updating the index's Dimensions setting will cause writes to silently skip indexing that particular item, since a dimension mismatch disqualifies it from the vector index entirely. You will not see an error, you will just notice later that some items never show up in search results.
- Treating filters like SQL WHERE clauses. Inline filters only support equality. Anything range like, for example "memories from the last seven days," needs to be modeled as a separate post filter step inside your application rather than something you try to push into SearchConditionExpression.
- Assuming immediate read after write. Because the index is eventually consistent, a memory you just wrote might not appear in the very next search a few hundred milliseconds later. Design your agent's short term working memory so it does not depend on that kind of instant visibility.
Taking this from a demo to a production memory store
Everything above works fine for a demo or a proof of concept, but a few extra considerations tend to matter once real traffic shows up.
First, think about time to live. Agent memories that are only relevant to a single support ticket or a single session probably should not live forever. DynamoDB's native TTL feature works on any table, vector index or not, so setting an expires_at attribute and enabling TTL on it is a simple way to keep the memory store from growing without bound.
Second, think about how many vector indexes a single table actually needs. A table can carry up to five vector indexes by default, which opens the door to having separate indexes tuned for separate purposes, say one scoped to session level recall and another scoped to organization wide recall across all sessions for a given customer. Just remember that each additional index adds its own write and storage cost, so this is a tradeoff rather than a free lever to pull.
Third, watch the inline filter list carefully as your schema evolves. Filters are locked in at index creation time along with everything else, so if you know you will eventually want to filter on, say, a priority field or a source field, it is worth adding that attribute to InlineFilterAttributes up front even if you are not using it yet, rather than having to stand up a second index later just to add one filter.
Fourth, consider how you handle updates to an existing memory. If an agent revises a fact it previously stored, you generally want to overwrite the existing item rather than append a new one, otherwise your recall results start returning stale and current versions of the same fact side by side with no way to tell them apart. Keying memories by a stable identifier tied to what they represent, rather than a fresh ULID every time, can help here depending on your use case.
Fifth, plan for multi region if your agent needs to run close to users in more than one part of the world. DynamoDB Global Tables replicate a table, and the data on it, across regions, and vector indexes participate in that replication the same way the base table does. This means an agent running in one region can benefit from memories written by an instance of the same agent running in another region, without you having to build a separate replication path for the vector data specifically.
Sixth, keep an eye on how backfill behaves if you add the vector index to a table that already has data on it. Because backfill runs at no additional charge and does not block reads or writes on the base table while it runs, adding vector search to an existing DynamoDB backed application is usually far less disruptive than people expect the first time they plan for it. Check the Backfilling flag on the index before relying on search results being complete, and treat the first few minutes after enabling a new index as a period where recall may be returning a partial picture of the data.
One table, real memory, and a lot fewer moving parts
Native vector search turns DynamoDB from "the operational store sitting next to your vector database" into the vector database itself, at least for workloads where your data already lives there in the first place. For an agent memory store specifically, that means one write path to reason about, one table to think about consistency and cost for, and search results that arrive with the full item already attached, with no second lookup required to make use of them.
The pattern walked through in this tutorial, embed then PutItem on the write side, embed the query then SearchVectors on the read side, then wrap the whole thing in a tool the agent can call, generalizes well beyond agent memory specifically. Product catalogs, support ticket similarity search, document heavy RAG corpora, and recommendation feeds all fit roughly the same shape, and the code above is a reasonable starting point for any of them.
None of this requires standing up new infrastructure, learning a new query language, or building a pipeline whose only job is keeping two databases honest with each other. That is really the headline change here. Vector search stops being a separate architectural decision you make on top of DynamoDB and becomes a property of the table you were already going to build, which means the interesting design work shifts from "how do we sync these two systems" to "what should our agent actually remember," which is a much better problem to spend your time on.
References
- AWS News Blog, Amazon DynamoDB now supports real-time vector search at any scale, https://aws.amazon.com/blogs/aws/amazon-dynamodb-now-supports-real-time-vector-search-at-any-scale/
- AWS Database Blog, Build semantic search with native vector support in Amazon DynamoDB, https://aws.amazon.com/blogs/database/build-semantic-search-with-native-vector-support-in-amazon-dynamodb/
- AWS Documentation, Using vector indexes in DynamoDB, https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/VectorSearch.html
- AWS Documentation, Tutorial, your first vector search, https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/VectorSearchTutorial.html
- AWS Documentation, SearchVectors API reference, https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API\_SearchVectors.html
- AWS What's New, Amazon DynamoDB now supports real-time vector search, https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-dynamodb-vector-search/
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article