Building an E-commerce AI Agent with AWS Serverless: Conversation Memory and Context Management (4/7)
Manage conversation history, persistent memory, context windows, and prompt caching to improve efficiency and reduce costs.
Series: Building an E-commerce AI Agent with AWS Serverless (7 articles)
- …
- 4Building an E-commerce AI Agent with AWS Serverless: Conversation Memory and Context Management (4/7) This article
Giving AI Agents Memory
Imagine a customer asks our shopping assistant to find running shoes and then follows up with “Which of those are waterproof?” An LLM doesn't remember previous requests. Each call is independent, so the agent needs the earlier messages to understand what “those” refers to.
Conversation history is the record of messages, including user questions, agent responses, tool calls, and their results. To continue a conversation after the agent restarts, we need to save this history and restore it later. This is called session persistence. It is different from long-term memory, which lets an agent retain useful information, such as customer preferences, across separate conversations. For our shopping assistant, continuing the same conversation matters more than remembering customers between chats.
Where we keep conversation history depends on the runtime. AgentCore Runtime can keep the agent and its history in memory during a session, but that session eventually ends, so we still need persistent storage. Lambda Event and Lambda Web Functions cannot reliably preserve history between requests, even when an execution environment is reused. Part 2: Choosing the Right Runtime compares these hosting options.
Saving the history is only part of the problem. Context is everything we send to the LLM for a particular request: instructions, tools, relevant conversation history, and other information. We don't need to send the entire saved history every time. As conversations grow, doing so becomes more expensive and can reduce response quality. Context management helps us choose what to keep, summarize, or leave out.
We'll first look at session persistence, then at AgentCore Memory—which calls stored conversation events short-term memory and also offers long-term memory—and finally at context management and prompt caching.
Session Persistence
Strands Agents provides session management. It can save and restore conversation history and agent state, allowing an agent to continue after a restart. We only need to configure persistent storage, such as Amazon S3.
Files and other storage options
There are several ways to keep an agent's state between requests:
Local files in the AgentCore microVM are easy to use for temporary work. They disappear when that compute environment stops, so they are not a reliable place for chat history.
Amazon S3 can store conversation history and other durable agent data. Strands Agents provides built-in S3 storage support, so we don't need to implement session persistence ourselves.
Amazon DynamoDB is another option for storing conversation history and session data.
An existing SQL database can store messages as rows if the shop already has one. This is familiar and easy to query, but adds database connections and schema work to the agent.
A shared mounted filesystem, such as Amazon EFS or S3 Files, is another option when several sessions need access to the same files. It requires more infrastructure and is more than this chat needs.
Persistent Session Filesystem
AgentCore Runtime creates a separate execution environment for each session. When the session stops, that environment is destroyed, including everything stored in memory and on its local disk.
Persistent Session Storage (currently in preview) solves the disk storage problem. It provides a persistent filesystem attached to the session. If the session stops and is later resumed, AgentCore creates a new execution environment and automatically reattaches the same storage, so previously saved files are still available.
Each session has its own isolated storage. Files are retained across restarts, but are deleted after 14 days without activity or when the agent runtime version changes.
This can be useful for storing conversation history, intermediate results, and other session data. However, it does not automatically preserve the agent's in-memory state. We still need to save that information to files and load it again when the session restarts.
AgentCore Memory
AgentCore Memory is a managed memory service for AI agents. It provides both short-term conversation memory and long-term memory that can extract and retrieve useful information across sessions.
Short-term memory
Short-term memory stores conversation events for a session and lets the agent reload them later. It is useful for resuming a conversation after the runtime stops or when the user returns to an existing chat.
For this use case, the advantage over S3 session storage is relatively small. Strands Agents already provides session persistence with S3, so we don't need AgentCore Memory just to save and restore conversation history. We can store messages at a much lower cost.
Long-term memory
Long-term memory is where AgentCore Memory provides capabilities that a normal database does not provide by itself. Instead of only storing raw conversations, AgentCore can process them and maintain a smaller set of useful memories across sessions.
- Automatic memory extraction — identifies useful facts, preferences and other information from conversations.
- Memory consolidation — updates or merges related memories and avoids keeping redundant or outdated versions.
- Semantic retrieval — searches memories by meaning and returns information relevant to the current question instead of loading all previous conversations.
- Memory strategies — supports different kinds of memory such as semantic facts, user preferences, summaries and episodic memory.
- Smaller LLM context — retrieves only relevant memories instead of sending a large conversation history to the model.
Building the same behavior with S3 alone would require additional application code, LLM processing and usually semantic-search infrastructure. AgentCore manages that pipeline for us. The processing can also be customized, including the instructions and Bedrock model used for extraction or consolidation, but the main benefit is that the memory workflow is already provided.
Cost and our choice
Amazon S3 is a very inexpensive option when we only need durable conversation history. AgentCore Memory costs more, especially when long-term memory processing and retrieval are used, but for many chatbot workloads the absolute memory cost can still be small compared with LLM inference.
For our shopping assistant, we only need to save conversations and restore them when customers return. Since Strands Agents already supports S3 session storage, S3 is the simpler and more cost-effective choice. We should configure an S3 Lifecycle rule to automatically delete old conversations after the chosen retention period, keeping costs low and avoiding indefinite retention. AgentCore Memory becomes more interesting if we later want cross-session preferences or other intelligent long-term memory.
We deliberately don't want long-term memory for this shopping assistant. Remembering customers across separate conversations could make them feel that we are tracking their behavior. We'd rather keep each chat focused on the current conversation and respect customers' privacy.
Context size
Every LLM has a limited context window — the maximum amount of information it can process in a single request. This includes the system prompt, conversation history, tool definitions, and tool results.
As a conversation grows, so does the context we send to the model. This creates two problems:
- Higher costs: More input tokens mean more expensive LLM calls, even with prompt caching.
- Lower response quality: With too much information, the model may struggle to identify what's relevant or overlook important details. We should therefore start reducing context before it reaches roughly 70–80% of the model's context window. This is a practical guideline, not a strict limit.
Context reduction techniques
There are several ways to reduce context without losing important information:
- Sliding window: Keep only the most recent messages and remove older ones. Simple and inexpensive, but the agent may forget something discussed earlier.
- Summarization: Replace older messages with a short summary. For example, a long discussion about running shoes could become: "Customer wants waterproof running shoes under €120 and prefers a wider fit." This preserves useful information while reducing token usage.
- Tool-result offloading: Large tool results, such as product search responses, can fill the context quickly. Instead of keeping them in every request, store them externally and retrieve them only when needed.
- Selective retrieval: Store older conversation information externally and retrieve only what's relevant to the current question.
- Pin important context: Always preserve essential information, such as instructions, product IDs, and decisions made during the conversation.
- Agentic context management: Let an LLM decide which information to keep, summarize, or remove. This is more flexible but introduces additional complexity and LLM costs.
Context management with Strands Agents
Strands Agents already provides conversation managers that help control context size.
The
SlidingWindowConversationManager keeps recent messages and removes older ones when the context becomes too large. It also supports proactive compression, so we can start reducing context before reaching the model's limit.The
SummarizingConversationManager takes a different approach. Instead of simply removing old messages, it uses an LLM to summarize them, preserving important information in a more compact form.For our shopping assistant, a sliding window is a good starting point. Most conversations should be relatively short, and keeping recent messages is usually sufficient. If we find that the agent loses important information during longer conversations, we can introduce summarization.
Prompt caching
As conversations get longer, the context grows. Every time the user sends a new message, we send the conversation history, system prompt, tool definitions, and other relevant information to the LLM again.
Without caching, we would pay the full input-token price for the same information on every request. Fortunately, most major LLM providers support prompt caching, which allows them to reuse previously processed context at a lower cost.
What gets cached?
Prompt caching typically works on the unchanged beginning of the prompt (prefix). This can include system instructions, tool definitions, previous messages, and tool results.
For example, when a customer sends their fifth message, most of the context is identical to the fourth request. The provider can reuse that cached portion and charge a lower price for those tokens.
There are a few limitations:
- Cached tokens are cheaper, but usually not free.
- Changing earlier content can invalidate the cache for everything that follows.
- Caches expire, typically after minutes or hours. Keeping stable instructions at the beginning of the prompt helps maximize cache reuse.
Caching depends on the provider
Not every LLM provider enables prompt caching automatically. Some handle it transparently, while others require explicit configuration.
- OpenAI: Prompt caching is automatic for supported models. No special configuration is normally required.
- Anthropic Claude: Through the direct API, prompt caching must be enabled using cache controls. Both automatic and explicit cache-point configuration are supported.
- Amazon Bedrock: Depends on the model. Some support implicit caching, while others require explicit cache points.
- OpenRouter: Depends on the underlying model and provider. Some support automatic caching, while others require explicit configuration. When using Strands Agents with Amazon Bedrock, we can configure automatic placement of cache points:
const model = new BedrockModel({ modelId: "...", cacheConfig: { strategy: "auto", },});With strategy:
auto , Strands determines where to insert cache points for the selected model. Without this configuration, Strands does not add explicit cache points, although Bedrock may still provide implicit caching.Prompt caching can significantly reduce the cost of longer conversations, but we should verify that it is enabled and actually working for our selected model and provider. It does not reduce the number of tokens in the context window, which is why we still need the context reduction techniques described earlier.
Series: Building an E-commerce AI Agent with AWS Serverless (7 articles)
- …
- 4Building an E-commerce AI Agent with AWS Serverless: Conversation Memory and Context Management (4/7) This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article