Why AI coding agents forget: memory, handoffs, and latent context
AI coding agents forget between sessions and at agent handoffs. How Claude Code, Copilot, Kiro, NVIDIA Dynamo, and recent research tackle each.
AI coding agents forget in two places: between sessions, when the lessons from your corrections don't stick, and within a session, when one agent hands work to another and the receiver starts from a summary. Both cost you the same way. You repeat yourself, and the agent repeats its mistakes.
This post looks at how production coding agents and recent research deal with each kind of forgetting. It's the companion to Build a self-improving development environment with Trace and Kiro , which shows how to build one answer to the first problem in Kiro.
Target audience: developers and architects building with AI coding agents or multi-agent systems. Learning level: 300 (advanced).
Two kinds of forgetting
| Across sessions | Within a session |
|---|---|---|
What's lost | Lessons from your corrections | What one agent worked out before handing off |
Where the knowledge lives | Rule files, steering files, memories | The sender's context and internal state |
Timescale | Days to weeks | Seconds |
Common approaches today | Agent memory; reviewed edits to rule files | Text summaries, transcripts, cache reuse |
Research frontier | Self-improving contexts, failure attribution | Cache and latent-state sharing between agents |
The two problems call for different tools, but they interact, as the last part of this post shows.
Across sessions: how coding agents learn from corrections
Several coding agents now learn from your corrections automatically. Here's how the main ones compare:
Agent | Rule files | Learns from corrections | Proposes edits to rule files |
|---|---|---|---|
| Claude Code | CLAUDE.md, .claude/rules/ , or AGENTS.md (read when there's no CLAUDE.md) | Yes, automatically (auto memory) | Yes ( /doctor prompt-audit ) |
| GitHub Copilot | copilot-instructions.md | Yes, automatically (memory shared across Copilot's agents) | Not documented |
| Windsurf / Devin (Cascade) | Rules | Yes, automatically (generated Memories) | Not documented |
| Gemini CLI | GEMINI.md | Only when asked ( save_memory ) | Not documented |
Cursor | .cursor/rules , AGENTS.md | Not verified | Rules can be generated from chat |
| Kiro | Steering files, AGENTS.md | No built-in feature | No |
Cursor's memory features couldn't be verified from its documentation; its rule generation from chat is described in community sources.
Two patterns stand out.
Memories pile up beside your rules, not in them. Auto memory, Copilot memory, and Cascade Memories capture what the agent learned, usually in a separate store or notes file. That's convenient, but the rule that caused the mistake stays wrong, and the memory store grows without anyone reviewing it.
Few tools close the loop back to the rule file. Claude Code's prompt audit comes closest: it produces a report of proposed edits to your instruction files, and nothing changes until you approve them. Amazon's Trace research (KDD 2026) goes further. It traces each failure to the specific source that caused it, classifies the problem as missing or stale content, verifies the diagnosis, and proposes a reviewable patch to that file. On 60 synthetic failure traces, it chose the right fix operation and the right file 82% of the time. The companion post shows how to build this with Kiro hooks and an MCP server.
Whichever route you take, two lessons apply:
- Edit in small steps. ACE (ICLR 2026) found that contexts rewritten over and over suffer context collapse: iterative rewriting erodes detail over time. Small, itemized edits are safer than regenerating a file.
- Put shared rules where every tool can read them. AGENTS.md is now an open format stewarded by the Linux Foundation's Agentic AI Foundation and read by many coding agents, including Kiro. Fixes written there carry across tools.
Within a session: the handoff cold start
The second kind of forgetting happens on a much shorter timescale: when one agent hands a task to another within a session.
In a multi-agent workflow — say, a research agent that maps a codebase hands off to an agent that implements a feature — the receiver typically gets a text summary. Two different things are lost at that boundary:
- Compute. The receiver re-reads context the sender already processed. On production inference stacks, much of this cost is already avoided by reusing the model's cache of processed context (the KV cache). NVIDIA measured Claude Code-style agent harnesses on Dynamo: once a prompt prefix is cached, later calls hit the cache 85–97% of the time, and 97.2% across a team of four agents. The same study captured the handoff cold start directly: teammate agents averaged a 79.4% hit rate against 91.3% for the lead agent's subagents, a gap driven almost entirely by each teammate's first call.
- Information. Whatever the summary leaves out — ranked alternatives, confidence, rejected hypotheses — is gone for good. Caching can't recover it, because it was never written down.
Five ways to hand off context
Production systems and research tackle the information problem, and sometimes the compute problem too, in several ways:
Handoff method | What crosses the boundary | Maturity | Examples |
|---|---|---|---|
Text summary or compaction | A natural-language summary or structured notes | Standard practice | Anthropic's context-engineering guidance |
Full or filtered transcript, plus cache reuse | The conversation history; the KV cache is reused when both agents run the same model | Production | OpenAI Agents SDK handoffs
, NVIDIA Dynamo |
Shared memory store | Extracted facts and memories | Production | GitHub Copilot memory,
Amazon Bedrock AgentCore Memory |
Cache or latent-state sharing between agents | The model's internal state | Research | KVCOMM
(over 70% cache reuse with no quality loss),
LatentMAS
(up to 14.6% higher accuracy, 70.8–83.7% fewer output tokens, about 4× faster),
cache alignment across different models |
Learned, compressed latent payload | A small learned summary of internal state, projected into the receiver | Research (6G radio) | ILCP |
What a 6G radio paper adds
The last row of that table comes from a different field entirely. Inductive Latent Context Persistence (ILCP; Banerjee and Awan, Nokia, AI4NextG @ ICML 2026) tackles the same cold start in 6G radio networks. When a phone moves between base stations, the target station's model normally restarts its per-device state from scratch. ILCP compresses the source model's 128-dimensional recurrent state into a 128-byte payload with a β-VAE (a variational autoencoder with a tunable compression bottleneck), sends it with the standard handover message, and adapts it to the target's state space with a learned gated projection. On 31 test handovers, ping-pong handovers fell from 6.5% to 0.0%, and next-cell accuracy improved by 5.1 percentage points on average (peak 13.3) in the 50–250 ms window after handover.
ILCP suggests that a small, learned summary of internal state can carry what text summaries lose. Its first author has published a reference implementation (source-available, all rights reserved) that adapts the mechanism to LLM agents, and is explicit that the telecom results are not agent results.
What to do today
Know the practical limit. Sharing cache or latent state requires access to the model's internals. It works with self-hosted open-weight models served by engines such as vLLM, SGLang, or LMCache. It doesn't work through API-hosted models such as Claude on Amazon Bedrock, and it isn't something you can configure in Kiro today. For Kiro users, the practical options are the first three rows of the table; the research rows show where handoffs are heading.
Use a structured handoff record. Recent theory on what a handoff must retain (Kato and Kato, 2026) argues that decisions and constraints should be passed on exactly. In practice, that means a structured handoff record — decisions, constraints, open questions, and pointers to evidence — rather than a free-form summary, with cached or latent state as an optional extra where your stack supports it.
For example, a research agent handing off to an implementation agent might pass:
Plain text
{"task": "Implement the order-export feature","decisions": ["Use an SQS FIFO queue for export jobs"],"constraints": ["Exports must finish within 15 minutes"],"rejected": [{"option": "Step Functions", "reason": "no operational runbooks yet"}],"open_questions": ["Do partner APIs need idempotency keys?"],"evidence": ["src/orders/export.ts", "docs/adr/0012-queues.md"]}
The
rejected and open_questions fields carry exactly what a free-form summary tends to drop. Write the record to a file the next agent reads, or pass it in the handoff prompt.Be careful with latent benchmark claims. Treat benchmark gains from latent methods with some care: a causal audit of three relay methods found that one released cross-model projector showed no detectable example-specific transfer.
How the two loops fit together
Trace and handoff methods fix context loss at different timescales:
Layer | Problem | Approach | Timescale |
|---|---|---|---|
Static context (steering files, specs, AGENTS.md) | Context rot: stale or missing knowledge across sessions | Trace: mine corrections, attribute them to files, and propose patches | Days to weeks |
Dynamic context (runtime state at agent handoffs) | Cold start: the receiving agent loses what the sender knew | Structured handoff records today; cached or latent state where the stack allows | Seconds |
The two loops could reinforce each other: better steering files give every agent a better starting point, and fewer losses at handoffs should mean fewer corrections. That's a hypothesis, not a measured result.
There's also a real tension. Trace works by reading the agent's reasoning; the paper notes that the agent's chain of thought is its key diagnostic signal, because it names the sources that shaped each decision. A handoff that passes opaque internal state instead of text would leave Trace less to read. Until latent handoffs can be inspected, keep a readable handoff record alongside any latent payload, so the attribution loop still has something to work with.
Open questions
- Access. Cache and latent-state sharing need access to a model's internals. They aren't available through API-hosted models or inside Kiro today.
- Auditability. Transferred state is opaque. It can't be reviewed the way a text handoff can, and attribution tools such as Trace can't read it.
- Security. The security of latent channels between agents is an open research question, as a recent survey of latent communication notes.
- Evidence. ILCP's results come from 31 handovers in a radio network, and its author describes agent benchmarks as future work. Cache alignment across different models works best when both agents see the same input.
Conclusion
AI coding agents forget in two places, and the fixes are maturing at different speeds. Across sessions, memory is now common; the next step is closing the loop back to the rule files themselves, with attribution and human review. Within a session, production stacks already avoid much of the recomputation at a handoff, but not the information a summary drops. Structured handoff records are the practical answer today; cache and latent-state sharing are where research is heading.
To build the across-sessions loop in Kiro, see Build a self-improving development environment with Trace and Kiro. For the handoff side, start with a structured handoff record like the one above, and see NVIDIA's notes on agentic inference with Dynamo for what cache reuse already saves.
References
- Zhao, Y., Misra, P. K., & Pandey, S. (2026). TRACE: TRajectory Attribution for Automated Context Engineering. KDD '26. arXiv:2608.09153
- Zhang, Q. et al. (2026). Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. ICLR 2026. arXiv:2510.04618
- Anthropic. How Claude remembers your project. Claude Code documentation
- GitHub. About agentic memory for GitHub Copilot. GitHub Docs
- Cognition. Cascade Memories. Devin documentation
- Google. Memory tool (save_memory). Gemini CLI documentation
- Linux Foundation. Linux Foundation Announces the Formation of the Agentic AI Foundation (AAIF). Press release · agents.md
- NVIDIA. Full-Stack Optimizations for Agentic Inference with Dynamo. NVIDIA Dynamo documentation
- Anthropic. Context engineering: memory, compaction, and tool clearing. Claude cookbook
- OpenAI. Agents SDK: Handoff. OpenAI Agents SDK documentation
- AWS. Build agents to learn from experiences using Amazon Bedrock AgentCore episodic memory. AWS Machine Learning Blog
- Ye, H. et al. (2025). KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems. NeurIPS 2025. arXiv:2510.12872
- Zou, J. et al. (2025). Latent Collaboration in Multi-Agent Systems (LatentMAS). arXiv:2511.20639
- Chen, S. et al. (2026). See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents. arXiv:2606.13594
- Liu, Y. (2026). Beyond Tokens: A Unified Framework for Latent Communication in LLM-based Multi-Agent Systems. arXiv:2606.05711
- Banerjee, A., & Awan, D. A. (2026). Inductive Latent Context Persistence: Closing the Post-Handover Cold Start in 6G Radio Access Networks. AI4NextG @ ICML 2026. arXiv:2605.00593
- Banerjee, A. (2026). ILCP-for-Agents (source-available reference implementation). GitHub
- Banerjee, A. (2026). Persistent Latent Memory for Multi-Hop LLM Agents. Towards Data Science
- Kato, M., & Kato, T. (2026). Handover of In-Context Learning State Across Session Boundaries. arXiv:2608.14528
- Cheng, J., Das, S., & Ramnath, R. (2026). When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs. arXiv:2608.04893
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article