Everyone Taught Their Agent to Dream. Nobody Taught It to Remember Last Tuesday ๐
Every agent memory product on the market is consolidation: many sessions crushed into a few durable rules. This is the other half, episodic retrieval where each Claude Code session stays its own searchable memory, built on S3 Vectors and Bedrock
Your assistant is great at consolidating rules and terrible at recalling occasions. Here's episodic memory for coding agents: retrieval where the corpus is your own prior reasoning traces, distilled on Bedrock, indexed in S3 Vectors, served to Claude Code and Cursor over MCP.
๏ปฟ
The bug you already fixed
Last month you lost an hour to a React list rendering stale rows after an optimistic update got rolled back. You suspected the reducer. You cloned state to prove it was mutating. It wasn't, because it never is, and yet we check every time. Eventually you diffed the rendered tree, traced the key derivation to a cached index map, and fixed it in four lines. Then you closed the tab with the serene confidence of a man who will absolutely remember this.
Next month a cousin of that bug lands in another repo. Your agent starts from zero. You start from zero. The reducer looks guilty again, because a stale render always looks like a mutation bug, and neither of you has any idea that Past You already ran this exact investigation and filed the report.
The report, incidentally, still exists. Every Claude Code session is logged to ย
~/.claude/projects/**/*.jsonlย : what you asked, which files it read, what it tried, what exploded, what finally worked. Full trajectory, sitting on your disk, doing approximately nothing.Everyone is building the same memory, and it's the wrong one
Here's the thing that made me build this. The whole industry has converged on one shape of agent memory, and it's consolidation: take many sessions, distil them into a few durable rules.
That's ย
CLAUDE.mdย . That's Cursor's memories. That's Claude Code's AutoDream. And that's ย Dreams on the Claude Platformย , which is the most serious version of the idea. A dream takes an existing memory store plus somewhere between 1 and 100 past session transcripts, and produces a new memory store with duplicates merged, stale entries replaced, and fresh insights folded in. The input is never touched, so you can read the output and throw it away if it came back weird, which is a genuinely nice property. The docs describe the pipeline as a synthesis pass over the inputs rather than an editor working on the store's text, and that phrasing is the whole ballgame: synthesis means many going in and fewer coming out. Anthropic borrowed the name from hippocampal consolidation, the thing your brain does overnight while deciding what was worth keeping. It's in research preview, it works, and I'm not here to argue with it.But notice what everybody built. Psychologists split long-term memory into two kinds. Semantic memory is facts and rules stripped of the occasion: "Paris is the capital of France," "this team uses conventional commits." Episodic memory is the occasion itself, kept whole: that Tuesday, that bug, that wrong turn you took at 3pm.
Every agent memory product on the market is semantic. We collectively taught our agents to sleep, and skipped the part where they remember anything that actually happened.
Which is a shame, because debugging is pathologically episodic work. "This team uses conventional commits" has never once helped me find a bug. "A stale render reads as a mutation bug, so the reducer looks guilty" would have saved me forty minutes, and it is exactly the kind of hyper-specific, situation-bound observation that any many-to-few merge is designed to throw away. Consolidation isn't failing at this. It's succeeding at its job, and its job is to delete precisely this.
So: one to one. Every session stays its own retrievable episode. Nothing gets merged into anything.
The paper that says this should work
The idea isn't mine, and the framing that rearranged my head belongs to ย RAG over Thinking Traces Can Improve Reasoning Tasksย by Arabzadeh, Ma, Min and Zaharia at UC Berkeley:
RAG isn't weak at reasoning. The corpus is just wrong for the job.
Documents are great for factual recall. "What's the default timeout on this SDK" is a document question. But reasoning doesn't want facts, it wants process: how a similar problem got worked through, which suspect looked guilty and turned out innocent, which check would have ended it in five minutes. That signal is in nobody's documentation. It's in your traces.
Their second contribution is T3 ("Transformation of Thinking Traces", ยง3.2), an offline rewrite of raw traces into a retrieval-friendly shape. The finding under it: representation matters roughly as much as retrieval does. How you write the memory down decides whether you'll ever find it again.
And the result nobody quotes, which is somehow the most interesting one: this can be cheaper. A compact retrieved trace shifts work from expensive output decoding into cheap input context, and T3 cut GPT-5's cost per query by around 15% while raising accuracy. Retrieved reasoning substituting for generated reasoning. That is not the usual RAG bargain, where you pay in tokens for the privilege of being slightly less wrong.
Expectations, before the timeline gets to you
Putting this above the fold instead of in a footnote, because you're going to see one number quoted without context by someone with a rocket emoji in their bio.
The famous +56.3% is competition mathematics, achieved on the weakest model in the lineup (Gemini-2.5-Flash climbing from 53.3 to 83.3 on AIME). GPT-5, already strong, gained 7.6% on the same benchmark. The paper states this plainly: the gains from transformation are largest for weaker inference models. Retrieval helps most exactly where the model needed the most help, which is intuitive and also the least exciting possible version of the headline.
On LiveCodeBench, the benchmark nearest to a coding corpus, gains land at +4 to 6%. And plain chunked raw traces beat the fancy T3 rewrite for two of the three models there. Your own repos are neither benchmark.
The bet is that recurring problems (your codebase, your patterns, your same three mistakes on a loop) repay retrieval better than one-off contest problems do. I think that bet is reasonable. It is also unproven, which is why the repo ships an eval harness instead of a victory lap, and why you should be skeptical of the guy who built the thing telling you the thing works.
What an episode looks like
Each session gets distilled into one card, roughly 150 words standing in for an hour of your life:
{ "problem": "A React list re-rendered stale rows after an optimistic update was rolled back.", "steps": ["Reproduced with a forced rollback in a test", "Diffed the rendered tree before and after", "Traced the key derivation to a cached index map", "Rebuilt the key map on rollback"], "key_decisions": ["Compared rendered trees rather than reading the reducer", "Treated key derivation, not state identity, as the suspect"], "insight": "Rollback restored the array identity but not the per-row memo keys.", "dead_ends": ["Suspected the reducer was mutating state, a structural clone changed nothing"], "misleading_intuitions": ["A stale render reads as a mutation bug, so the reducer looks guilty"], "critical_checks": ["Check whether child keys changed, not just parent array identity"], "outcome": "verified", "tags": ["web", "react", "rendering"]}Note what's absent: no code, no file paths, no stack traces. Those are incidentals. Next time the file will have a different name and the line numbers will have wandered off, but "the reducer looks guilty" will still be true, and still wrong.
ย
dead_endsย and ย misleading_intuitionsย are the fields I'd defend with a chair. Knowing where not to look is half of debugging, and it's the half that never survives contact with a summary. Nobody has ever written "spent 40 minutes barking up the wrong tree" in a commit message. The commit says ย fix: rebuild memo keys on rollbackย , which is true, useless, and a small act of historical revisionism.The three field groups aren't decoration either. The paper defines three ways to rewrite a trace (ย
structย , ย semanticย , ย reflectย ) and which one retrieves best depends on the domain, so one distillation emits all three and you go find out which.So how is this different from Dreams, or Cursor's memories?
The question everyone asks, so let's do it properly. Four axes.
What survives. Consolidation keeps the generalization and drops the texture. That's the point of it. But texture is the payload here: "a stale render reads as a mutation bug" is worthless as a durable rule and priceless the moment you're staring at a stale render. Merge it into anything and it evaporates.
How it scales. Consolidation has a ceiling by construction. A dream takes at most 100 sessions. Memory files bloat, contradict themselves, and go stale, which is why AutoDream exists to prune them. An episodic corpus has no ceiling, because you never load it. You retrieve three cards. At 800 sessions, consolidation hands you a thinner summary; retrieval hands you the one Tuesday that matters.
What you pay for. Dreams bills at standard token rates and, per the docs, cost scales roughly linearly with the number and length of the input sessions. Which makes sense: every dream re-reads the transcripts. Distillation here is one-shot and content-hashed, so a session is read exactly once in its life, and every retrieval afterward is an embedding call and an S3 Vectors query. Consolidation is a recurring bill against your whole history. Episodic indexing is a one-time bill per episode. Over a year of sessions those two curves go to noticeably different places.
Whose computer. Dreams runs in Managed Agents, on agents you've built on the Claude Platform. Cursor's memories run in Cursor. This runs in your AWS account, serves Claude Code and Cursor off one index, and your traces never leave. If your sessions contain client code, that stops being a preference and becomes a requirement.
Now the part where I argue against myself, because Dreams has one property I don't: the output is a separate store you can read, diff, and discard before adopting. A human gets to look at the consolidated memory and veto it. My cards go straight into the index, and my defenses are automated (verified-only search, outcome judged on evidence in the trace). That's weaker supervision, and I'd rather say so than have you discover it.
And the real answer: these are complementary, not rivals. Consolidation gives your agent "how this team works." Episodes give it "how this kind of bug tends to behave." Semantic and episodic memory coexist in your skull for excellent reasons. Run both. I'm not trying to talk you out of Dreams. Dreams and I are solving different halves, and it took me an embarrassingly long time to notice that was even the argument.
(Also worth saying out loud: AutoDream now reads the same JSONL transcripts this harvests. So the line about those files sitting there doing nothing was true when I started building and is aging in real time. The distinction holds, since it consolidates them rather than indexing them, but "nobody reads your session logs" is no longer one of my selling points. Software moves fast. Blog posts age like milk.)
The pipeline
๏ปฟ
Two paths, and only one of them ever makes you wait.
The write path is event-driven and entirely out of your way. Uploading a transcript to S3 is the trigger; there's no separate index step to forget to run. PutObject fires a Lambda that recovers the trajectory (request, files read, tool calls, edits, failures, fixes), scrubs secrets before anything leaves your account, distils the card with Sonnet on Bedrock under a JSON schema, embeds it with Titan v2 at 1024 dimensions, and writes it into an S3 Vectors index. A nightly cron running ย
harvestย keeps the corpus fresh. Nobody is waiting on any of this, least of all you.The read path is the only one that pays for latency, which is why it's the only thing kept warm. Claude Code and Cursor both speak MCP over HTTP, so the query side is API Gateway with bearer auth in front of a Lambda exposing a ย
search_tracesย tool, and your assistant calls it like any other tool. Claude Code can go further with a ย UserPromptSubmitย hook that injects relevant cards automatically, with a 1.5s timeout that ย exit 0ย s silently down every failure path, because a memory system that blocks your prompt is worse than amnesia. (Cursor can't do this. Its ย beforeSubmitPromptย ย quietly drops the modified promptย . I found that out the way you'd expect, which is slowly.)Everything model-facing goes through Bedrock rather than ย
api.anthropic.comย , for the deeply unromantic reason that AWS credits cover Bedrock. Ten-minute trap included free: Bedrock model IDs want the ย anthropic.ย prefix, and the bare ID just 400s at you with no particular sympathy.Why S3 Vectors. It went ย GA recentlyย , and for a corpus that grows by a few hundred cards a month and gets queried a handful of times an hour, running OpenSearch would be like renting a warehouse to store your socks. Darryl Ruggles did ย the actual arithmetic across S3 Vectors, OpenSearch, pgvector and Pineconeย , worth reading before you pick. Everything here stays in credit-covered territory except provisioned concurrency (~$5 to 15/mo), and that's one flag from zero: ย
-c provisionedConcurrency=0ย .Evaluate before you backfill. Please.
The one instruction in the README I would put on a billboard.
The card format is the variable that actually matters, and picking it after distilling 500 sessions means paying twice to find out you were wrong. So run the harness on a sample first:
npm run backfill -- --bucket <b> --limit 60 --out-dir ./eval/cardsnpm run eval -- --cards-dir ./eval/cards --concurrency 4For each card the harness writes a held-out query paraphrasing the symptom, with none of the solution vocabulary, because querying a card with its own words proves nothing except that cosine similarity is still doing its job. Then it embeds everything in all three strategies and reports recall@1, recall@3 and MRR. Recall@3 is the number that matters. A margin inside two standard errors gets reported as a tie rather than a winner, a small piece of statistical hygiene I'm quietly proud of and that nobody will ever notice.
ย
structย is the default, on the strength of the LiveCodeBench column. That's a prior, not a result about your corpus. Go find out.The gotchas
Embedding parity is the silent one. Store and search must agree on model, dimension and card format. A mismatch doesn't throw, it returns plausible noise, which is the worst failure mode in software because everything looks fine and is not. Synth refuses if ย
queryFormatย isn't in ย indexFormatsย , ย embed()ย throws on dimension mismatch, and nothing whatsoever catches a format mismatch at runtime. Yet.Poisoning is the real risk of the entire idea. A confidently retrieved wrong episode is worse than no episode, because now your agent begins from a bad hypothesis wearing a confident expression. Defenses: search is verified-only by default, and the distiller judges ย
outcomeย on evidence present in the trace rather than on vibes. The local heuristic discounts hedging, because "should be fixed now" is not a synonym for fixed. Anyone who has read their own commit history knows why that check had to exist.Context budget: 2 to 3 cards. Over-retrieval crowds out the actual task and makes results worse. Not just my hunch: the paper ablated k at 1, 3 and 5 and settled on 3, with k=1 missing complementary steps and k=5 hauling in noise. More memory is not better memory, a lesson available in several other areas of life.
Your traces are your source code. Private bucket, SSE-KMS with a customer-managed key, bearer auth, ย
BLOCK_ALLย , TLS enforced. Never put this behind a bare public Lambda Function URL. I resent having to write that sentence and I'm writing it anyway.Thinking text is mostly not there. In web and remote sessions, thinking blocks persist with the text stripped. I verified it on this repo's own logs: 10 blocks, 0 with text, a beautifully clean result for a hypothesis I was hoping to disprove. Not a blocker, because the useful signal for coding work is the trajectory and that's in the JSONL regardless. Thinking text is a bonus when it shows up. ย
npm run inspectย reports what your local CLI does.Try it without spending anything
None of this touches AWS or costs a cent:
npm installnpm run selftest # pure-logic checksnpm run inspect # survey ~/.claude/projects, estimate backfill costnpm run inspect -- --file <session>.jsonl --render # exact distiller inputRun ย
inspectย first. It tells you how much history you're sitting on and what a backfill would cost before you commit to anything. If the answer comes back "eleven sessions," this isn't for you yet. Come back in six months, you'll have plenty, and probably the same three bugs.Why I think this matters
Every serious debugging session produces two artifacts. There's the fix, which gets committed, reviewed, and remembered forever. And there's the method: the wrong turns, the suspect who looked guilty, the check that would have ended it in five minutes. The second one is worth more, because a fix solves exactly one bug while a method generalizes. It's also the one we delete every single time, usually within the hour.
We've spent two years teaching agents to consolidate: crush a hundred sessions into a handful of durable rules and call it memory. That's genuinely useful and I use it daily. But consolidation is lossy on purpose, and the specific texture of how this kind of problem goes wrong is exactly what gets averaged into nothing. You don't just need an agent that knows your conventions. You need one that remembers last Tuesday.
Whether it works on your corpus, I don't know yet. That's not modesty, that's the eval harness. Run it and find out, and please tell me if it doesn't, because that's the more interesting paper.
Repo: ย github.com/wakeupmh/rag-on-thinking-tracesย . Paper: ย arxiv.org/abs/2605.03344ย , with ย
docs/paper-notes.mdย covering the results table and every place I wandered off from it.Sleep is for consolidation. May your dead ends be indexed. ๐ป
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article