Everyone blames the model for the AI bill.
Wrong suspect.
Most tokens aren't spent thinking. They're spent re-learning what the agent knew yesterday.
Microsoft's bet makes it obvious
Microsoft is putting agents everywhere: the IDE, Office, Windows, all wired together through open protocols like MCP.
Not one assistant. Many agents, always on.
That's where forgetting gets expensive.
The leak
Every new session, the agent starts from zero:
- Re-reads the codebase.
- Re-maps how it connects.
- Re-learns decisions your team already made.
Same context. Paid for again.
A hypothetical team with 10 agents, 5 sessions a day and 200K tokens of context each is spending 10 million tokens a day just to get back to where it was.
Worse, ten agents rebuilding context separately means ten different versions of the truth.
Bigger windows won't save you
A bigger context window just lets you re-send more of the same thing.
That's a bigger leak, not a fix.
The fix
Treat context as infrastructure, not a prompt:
- Index once.
- Share across every agent.
- Refresh only what changed.
- Keep the why, not just the code.
Same models. Fewer tokens. No drift.
Stop asking "which model should our agents use?"
Start asking "how much are we paying them to forget?"
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article