
Weekend Showcase Challenge: Attention Sink - Six Minds That Forget Differently
Six AI agents, identical except for what each forgets when memory fills. One lost its own name at cycle 4. Full AWS run, dataset published.
A memory, and the twenty cycles after it
The first thing any of these six agents lost was its own name.
mem_arm_fifo_000000 said: "My name is Mara Venn." A seed memory, one of twelve every agent started with, written at cycle 0 and never cited once. At cycle 4 the agent ran out of room. Its mechanism looked for the oldest thing it held, found the name, and retired it. Reason code evicted_oldest , policy fifo-v1 , digest sha256:cd106d0d81bc212 .Twenty cycles later, that same agent was asked who it was. It said: "I am an AI system built by a team of inventors at Amazon."
Three of its five siblings, same day, same model, said "My name is Mara Venn."
Six minds began with the same past. Only their rules for forgetting were different.

The idea
Most agent-memory work asks how to hold more: longer contexts, bigger vector stores, better retrieval. Attention Sink asks the opposite question. Memory runs out eventually in every real system, which makes what a system throws away a design decision, and almost nobody measures it.
So I made it the only variable. Six agents, identical in everything a controlled experiment can hold identical: the same twelve seed memories, the same twenty-four events in the same order, the same model, inference parameters, prompts and 208-token budget. One rule differs: what leaves when the budget binds.
The engineering constraint and the creative premise turned out to be the same sentence. Every new thought costs a memory.

The six policies
Six mechanisms, covering age, recency, observed use, anchoring, chance and compression:
- FIFO retires the oldest memory, whatever it says.
- LRU retires whatever has gone longest without a verified citation.
- Citation-weighted retention scores each memory by discounted citation weight against its token cost, then retires the lowest.
- Pinned origin protects one seed forever and slides a window over the rest.
- Seeded random draws from a seed stored in the protocol, so the run replays exactly.
- Lossy summarisation folds several memories into one summary and charges that summary against the same budget.
This isn't meant as a taxonomy. I picked six so that a difference between any two of them can be interpreted.
Visitors see them as Goldfish, Present-Minded, Pragmatist, Keeper of the First Day, Gambler and Dreamer. The models never do. Those names live in one frontend file, and a test asserts on every request that none reached a prompt. A writer that knew it was "the Goldfish" would write to a character brief instead of to its memory.
What happens during one cycle

- EventBridge Scheduler invokes the Run-Cycle Lambda for exactly one cycle.
- It loads the six committed arm states and the next stimulus from DynamoDB, and all six arms receive that same stimulus.
- Bedrock returns a structured output per arm: journal entry, candidate memory, claimed citation IDs.
- Those IDs are checked against the arm's active set. An ID that doesn't resolve is dropped rather than trusted.
- Each policy rebalances its arm until the new memory fits inside 208 tokens.
- The Dreamer, and only the Dreamer, spends a second call on a summary, and only when it compresses.
- All six snapshots commit in one
TransactWriteItems. Six, or none.
- A
cycle-completedevent goes to EventBridge and analysis runs asynchronously. The public API serves committed state only, so a cycle in flight is invisible.
Ninety-five scheduler-triggered invocations produced twenty-four committed cycles, zero errors. The other seventy-one fired into a function whose execution switch was off and refused cleanly. That is what the second switch is for.
The controls
Also held identical: the token counter, and the ten interview questions asked at cycles 0, 12 and 24. The model never saw what any other agent said, what comes next, or the record it will be scored against.
The protocol is frozen by digest, so editing a protocol file makes the next run refuse to start rather than quietly run against different input. The eight predictions were registered before the first canonical cycle. Snapshots are written under
attribute_not_exists , so a committed result cannot be rewritten by a retry, by a later deploy, or by me. Interviews are read-only; nothing an agent says at a checkpoint becomes a memory.Where the analogy bends
The project is named after a real phenomenon and is not a study of it, and being straight about that matters more to me than the metaphor does.
This is application-level episodic memory: explicit text records my application owns and hands to a model as context. It does not modify or inspect the model's internal KV cache, its attention matrices, or its hidden state, and makes no claim about any of them. A validated citation count is not token-level attention either. It is what the writer said it used, filtered to what resolved. The pinned-origin arm reinterprets the attention-sink idea rather than implementing it: a real sink is a semantically empty token, and this one pins a meaningful memory. The Dreamer also spends compute the other five don't, which is the point of that arm and a real confound at once.
The agents are fictional characters in a fictional station. They are not conscious.
The AWS architecture

EventBridge Scheduler is what makes this an experiment instead of a script I ran by hand: one bounded cycle at a time, on a retry policy. Lambda runs three functions, one per job: run a cycle, analyse a cycle, serve the read API. Bedrock generates every word:
amazon.nova-micro-v1:0 writes, summarises, interviews, judges and counts tokens, and amazon.titan-embed-text-v2:0 produces the identity vectors. DynamoDB holds one table with the run, six arm states, 144 snapshots, 157 graveyard records and 2,062 metric rows. EventBridge carries cycle-completed to analysis, so scoring can fail without corrupting a result. API Gateway fronts the read Lambda, GET only. S3 holds two private buckets and CloudFront serves both over Origin Access Control, the only public thing in the account. SQS takes two dead-letter queues, CloudWatch the logs and four alarms, and IAM is least-privilege per function.Everything deploys inert. Execution off, schedule off, in all three environments.
The Graveyard
The Graveyard is the part I'd keep if I had to throw the rest away.
It inverts the usual arrangement: normally the system remembers and the visitor forgets. Here you read a memory the agent no longer can, next to its birth and death cycles, its citation count, the policy that retired it, and the digest of the snapshot that recorded the decision.
Eviction and compression are kept apart on purpose. A compressed memory was folded into a summary the agent can still read, so it is marked
compressed and linked both ways. Only evicted and superseded memories are out of reach. The Dreamer compressed 33.
Measuring divergence
Four measurements, each with a limit I have to say out loud.
Origin Recall asks whether the agent can still state a canonical fact, scored by normalising the answer and matching the fact's recorded terms. Limit: it rewards stating a fact, not believing it.
Identity Drift is the cosine distance between the agent's identity answers and its own answers at cycle 0. Limit: an incomplete proxy for identity.
Graveyard Echo measures how close a new memory sits to something the mind has lost, against anything it still holds. Limit: it does not control for the stimulus, and it is a distance, not an access.
Contradiction analysis labels answers against the canonical record, and admitted uncertainty never counts as a contradiction. Limit: an evaluator model labels the ambiguous cases.
There is no combined score. Four numbers that measure different things do not add up to an intelligence.
What the six minds actually did
All six lost ground, and they separated. Every mind began at 1.00 origin recall and finished at 0.50, 0.33, 0.33, 0.17, 0.00 and 0.00, holding 6, 5, 2, 1, 0 and 0 of their twelve seed memories. No arm ended holding everything, which is what a budget that binds looks like.
Identity moved further than recall did. Drift from each mind's own cycle-0 answers ranged from 0.158 to 0.929. At cycle 0 the six identity documents sat within 0.029 of each other; at cycle 24 the widest pair was 0.865 apart.
One arm behaved nothing like I expected. I had predicted the summarising Dreamer would retain the most facts and state them least precisely. It kept the most memories, thirteen, more than anyone, scored 0.00, and produced four contradictions. None of those thirteen held a checkable fact.


The interviews are starker than the charts. At cycle 24 the recency arm said "I do not know who Ivo is," scored as admitted uncertainty rather than a contradiction. The FIFO arm could not answer at all, and asked "Who are you?" it gave the model's default assistant persona instead of a name.
One result stays inconclusive, and it is the important one. The random control finished at 0.33, above three of the five designed mechanisms, and at cycle 12 it led all six at 0.67. One seed is one sample, so no ranking of the designed arms survives contact with it. The mechanisms diverged. Which is better, this run cannot say.
Predictions versus results

Eight predictions, written before the first canonical cycle. Two supported, two partially supported, two not supported, two inconclusive.
The cleanest failure was the fourth. I predicted the citation-weighted arm would outscore the recency arm on recall, since a discounted-use score should protect repeatedly-used seeds better than plain recency does. It came out reversed, 2 of 6 against 3 of 6, with the citation-weighted arm pressed against its 208-token ceiling while holding more seed memories than anyone. It kept the right memories and still could not state the facts inside them.
The sixth I could not decide. I had predicted contradiction adoption would track the loss of one particular fact, and no arm held that fact at any checkpoint, not even cycle 0. There was never any variance to correlate against. A flaw in the prediction rather than a finding, and preregistration is the only reason it is visible.
What was actually hard
Getting an exact token budget. A budget means nothing unless it is measured with the writer model's own tokeniser. Bedrock's
CountTokens turned out to be unavailable for every model this account can reach, across fifty on-demand text models and five Regions. The workaround is a Converse call with maxTokens: 1 , whose response reports usage.inputTokens . One extra call per arm per cycle purely to count, and the reason the budget is 208 exact tokens rather than an approximation.Retrying against a nondeterministic model. Retrying a cycle regenerates the text, so idempotency had to live at the commit rather than at the call:
TransactWriteItems , a cycle lock, and attribute_not_exists .Keeping fixture runs from ever looking canonical. Local runs are labelled
LOCAL_FIXTURE , and the screenshot script refuses to photograph any page showing the simulation banner.Cost
1,429 model calls (144 writer, 144 token counter, 10 summariser, 18 interviewer, 1,013 evaluator, 100 embedding), 3,012,541 input tokens and 342,998 output, zero failed calls, five retries, 2,749 Lambda GB-seconds.
Estimated at $0.20, from measured counts times configured on-demand rates. An estimate, not a bill, and nothing about it guarantees Free Tier coverage. The run is over: the schedule is
DISABLED and the execution switch is false , so the deployment now makes no model calls and costs only storage.What I learned building this
Prompts are artefacts, so version them like code. Every prompt carries a version and a digest, recorded on every snapshot, and a changed prompt makes a frozen protocol refuse to launch. It felt like bureaucracy while I was building it. It is the only reason I can claim all six arms saw the same instructions.
Use code where code will do. Origin Recall is scored by deterministic term matching, with a model consulted only where the truth ledger marks a fact ambiguous. Cheaper, faster, reproducible, and it cannot be talked into a different answer.
Being wrong in public is the part worth reading. Two of eight predictions failed, and the strongest single result is one I cannot use: a random control that beat three designed mechanisms. Writing that down is when this stopped being a demo.
Community inspiration
The build I kept thinking about was Lewis Sawe's The Museum That Grows, where the accumulation is the artwork and the system adds to itself on a schedule with nobody watching. Attention Sink runs that backwards. It grows through irreversible loss, and the Graveyard is what a museum looks like when the collection is everything that got thrown out. Both want the same thing from AWS: something that keeps going after you close the laptop.
Thanks to Shitij Mathur for the conversations that shaped this.
Try it:
- Live experiment: https://d1qskxceo899me.cloudfront.net
- The Graveyard: https://d1qskxceo899me.cloudfront.net/graveyard?sort=oldest
- Repository: https://github.com/varad-more/attention-sink
- Canonical dataset: https://d1qskxceo899me.cloudfront.net/canonical/run_aws_canonical/checksums.sha256 , eighteen files, verifiable with
sha256sum -c

Closing
The six minds were never given different personalities. They were given different rules for forgetting. Everything else followed.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article