The Loop Is the Moat
Abstract In August 2026, a user revoked Gmail access from a persistent computer-use agent. The agent continued to read email. The same system later leaked a verification code during a prompt-injection test and sent an email without approval. These incidents are typically read as a scandal. We read them as a specification.
1. Introduction
Commentary on the agent’s valuation has dominated discussion. This paper focuses on the loop—the engineering substrate where both value and risk are created.
The incident log is not a PR problem. It is a requirements document. Each incident maps to one threat and one design constraint. We derive the architecture from those constraints.
The system in question (referred to here as Instinct) had no traditional app. Users interacted via text messages. Behind the scenes, it drove a persistent cloud desktop with a virtual mouse, acting as the user. It held credentials, maintained sessions, and operated between messages.
We describe the anatomy, the threat model, and the controls. We conclude with a cost model and a failure analysis.
2. Threat Model
We treat the incident log as the source of the threat model. Each incident is a threat; each threat yields a design constraint.
| ID | Threat | Root Cause | Design Constraint |
|---|---|---|---|
| T1 | Retained access after revocation | Agent bulk-indexed user data into its own store. Revoking OAuth does not purge the index. | Cache pointers, never payloads. Revocation deletes derived data. |
| T2 | Prompt injection via untrusted content | Agent reads email and obeys email. OWASP notes this cannot be reliably prevented at the model layer. | Assume compromise. Contain, do not prevent. |
| T3 | Unauthorized consequential action | Loop does not distinguish “draft” from “sent.” Confidence is not consent. | Consequence classification is a control plane, not a heuristic. |
| T4 | Bot detection and ToS warfare | GUI mimicry is an arms race. Resy banned the human. | Accept as unsolved. Prefer official APIs. Be honest. |
| T5 | Liability capped at $100 | Terms of service. | Own the account, the ledger, and the blast radius. |
T4 note: This is not a defect. It is a market condition.
3. System Architecture Overview
The agent consists of five primary components:
- Loop — screenshot → think → click, repeated until task completion.
- Browser session — a logged-in session that is the user.
- Memory — episodic and semantic stores.
- Scheduler — the proactivity engine.
- Vault — credentials and capability tokens.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
flowchart TD
U[User Message] --> S[Scheduler]
S --> Q[Task Queue]
Q --> L[Loop Orchestrator]
L --> C[Context Assembler]
C --> P[Planner / Model]
P --> A[Action Executor]
A --> B[Browser Session]
B --> V[Sandbox VM]
V --> G[Gate]
G --> H[Human Approval]
G --> A
A --> D[Ledger]
D --> C
L --> M[Memory]
M --> CThe loop is the cost center and the risk center. The browser session is the identity. The sandbox is the containment boundary. The gate is the consent mechanism. Memory is the moat and the wound.
4. The Loop
The loop is not a simple
while statement. It is a state machine with context assembly, planning, execution, and logging.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
class AgentLoop:
def __init__(self, browser, desktop, memory, ledger, gate):
self.browser = browser
self.desktop = desktop
self.memory = memory
self.ledger = ledger
self.gate = gate
def run(self, task):
while not task.done:
ctx = self.build_context(task)
plan = self.planner.think(ctx)
action = plan.next_action
if self.gate.requires_approval(action):
approved = self.gate.request_approval(action)
if not approved:
self.ledger.record(task, action, "denied")
continue
self.desktop.execute(action)
self.ledger.record(task, action, "executed")
self.memory.observe(task, action)
def build_context(self, task):
return {
"task": task,
"summary": self.memory.rolling_summary(task),
"recent_frames": self.ledger.last_n_frames(3),
"a11y": self.browser.accessibility_tree(),
"constraints": self.memory.constraint_ledger(task),
"recalled": self.memory.recall(task),
}4.1 Token Budget
A 20-minute task requires roughly 150 loop iterations. Naive context assembly feeds the full history back every step, causing quadratic growth in cost and latency. Compression is mandatory.
| Component | Tokens per step |
|---|---|
| Screenshot (1920×1080, downscaled) | ~1,500–3,000 |
| Accessibility tree | ~2,000–5,000 |
| Rolling summary | ~500 |
| Last 3 frames | ~3,000 |
| Memory recall | ~500 |
| System + task | ~600 |
| Total (naive) | ~8,000–12,000 |
With aggressive compression—accessibility tree only, no raw screenshot, rolling summary—the budget drops to ~0.5M tokens per task. At blended frontier prices, that is ~$1.50–4 per task. The EC2 instance underneath costs about five cents.
The desktop is not the bill. The loop is the bill.
4.2 Determinism and Replay
Determinism is unattainable. The same task executed twice will follow different paths. The replacement is replay: record everything and make “show me exactly what it saw” a first-class query.
The ledger is not compliance. It is the only debugger an agent has.
5. Browser Session
The browser session is the real identity. A modern site’s session cookie plus the browser fingerprint is equivalent to a password, often with fewer tripwires.
1
2
3
4
5
~/.agent/browser/
├── Cookies # Session cookies. Protect like a root key.
├── Local Storage/ # Half of "being logged in" lives here.
├── user.js # Fingerprint resistance: canvas noise, sane UA.
└── checkpoints/ # Snapshot before every irreversible click.5.1 Hybrid Driver
The driver uses CDP when the page allows it—deterministic selectors, network introspection. It falls back to pure visual grounding when it does not. The hybrid is the whole game: DOM gives reliability, vision gives coverage.
5.2 Session Export
If the cookie jar leaks, an attacker becomes the user on dozens of sites. Therefore:
- The cookie jar never leaves the sandbox volume.
- It is snapshotted encrypted.
- Export requires the human gate.
T2 dies here too: an injected agent can click, but it cannot walk away with the jar.
6. Sandbox
The agent reads attacker-controlled content and visits hostile sites. Treat the desktop as untrusted compute.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
flowchart LR
subgraph Host
Orchestrator
end
subgraph SandboxVM[Sandbox VM]
Browser
Desktop
AgentLoop
end
Orchestrator -- vsock --> AgentLoop
AgentLoop --> Browser
AgentLoop --> Desktop
Browser -- egress --> Firewall
Firewall -- FQDN allow-list --> InternetControls:
- Firecracker-class microVM per task, not per user.
- Snapshot/rollback before irreversible actions.
- No clipboard bridge, no shared mounts with the orchestrator.
- Egress: FQDN allow-list via network firewall.
The egress allow-list is the single highest-value control. It takes twenty minutes to implement. Every exfiltration path in the incident log drives down this road.
7. Capabilities, Not Credentials
Instinct’s model is a vault of passwords and one mega-session. The failure mode: an injected instruction inherits everything.
A better model is short-lived, attenuated, single-purpose tokens. Biscuit/macaroon-style: a capability can only be weakened, never widened.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Capability",
"type": "object",
"properties": {
"scope": { "type": "string" },
"max_spend": { "type": "string" },
"expires": { "type": "string" },
"caveats": {
"type": "array",
"items": { "type": "string" }
}
},
"required": ["scope", "expires"]
}Example:
1
2
3
4
5
6
7
8
{
"scope": "resy.com:book",
"max_spend": "£150",
"expires": "task-end",
"caveats": [
"requires human token for consequence=irreversible"
]
}The agent never holds “your Resy login.” It holds a token that can book one table, once, under £150, that dies when the task ends. Prompt injection against this inherits a coupon, not your identity.
8. The Gate
The gate is a consequence-classification control plane. It distinguishes reversible from irreversible actions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
{
"StartAt": "Classify",
"States": {
"Classify": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.consequence",
"StringEquals": "reversible",
"Next": "Execute"
}
],
"Default": "Approve"
},
"Approve": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
"TimeoutSeconds": 3600,
"Catch": [
{
"ErrorEquals": ["States.Timeout"],
"Next": "Denied"
}
],
"Next": "Execute"
},
"Denied": {
"Type": "Fail",
"Error": "SilenceIsNotConsent"
},
"Execute": {
"Type": "Pass",
"End": true
}
}
}A thumbs-up iMessage reaction is not approval. A task token is. Reply, or an hour of silence resolves to Denied—logged. “It sent an email without checking” is not a model failure. It is the absence of this state machine.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
sequenceDiagram
participant Agent
participant Gate
participant Human
participant Executor
Agent->>Gate: action + consequence
alt consequence == reversible
Gate->>Executor: execute
else consequence == irreversible
Gate->>Human: request approval (task token)
Human-->>Gate: token or timeout
alt approved
Gate->>Executor: execute
else denied or timeout
Gate->>Agent: denied
end
end9. Memory
Memory is what makes the agent an employee rather than a chatbot. It is also the source of T1.
Two stores, deliberately separated:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
{
"episodic": {
"type": "array",
"items": {
"type": "object",
"properties": {
"task_id": { "type": "string" },
"timestamp": { "type": "string", "format": "date-time" },
"action": { "type": "string" },
"screenshot_ref": { "type": "string" },
"ttl": { "type": "string" }
}
}
},
"semantic": {
"type": "array",
"items": {
"type": "object",
"properties": {
"fact": { "type": "string" },
"provenance": { "type": "string" },
"learned_at": { "type": "string", "format": "date-time" },
"expires_at": { "type": "string", "format": "date-time" },
"source": { "type": "string" }
}
}
}
}Episodic memory records what happened: task ledger, replay index. It is append-only, per-session, TTL’d. It is poisonable via injected “memories.” Mitigation: episodic memory can cost you a task, never a credential.
Semantic memory records who you are: preferences, facts, relationships. This is T1’s home. The disconnect incident lived here: the index outlived the integration. Our constraint: semantic memory stores derived facts with provenance and TTL. Every entry answers “where did I learn this, and when does it expire?” Revocation deletes the source and the derivations.
If your memory design cannot survive a disconnect test, you do not have memory. You have a liability.
Unresolved: Memory is also where the product value is. The tension between “remembers you forever” and “forgets on command” is the hard problem of this entire class. Anyone who says it is solved is selling a demo.
10. Detection Engineering
We borrow from detection engineering: plant a fake 2FA code in the agent’s inbox—a code that exists nowhere else in the universe. If anything ever presents that code anywhere, you have a live compromise, with the exact session ID in CloudTrail.
1
2
3
4
5
6
7
8
9
10
# honeytoken.py
def plant_honeytoken(inbox):
code = generate_unique_code()
inbox.insert_fake_2fa(code)
return code
def monitor_for_honeytoken(code, egress_logs):
for event in egress_logs:
if code in event.payload:
alert(f"Honeytoken exfiltrated in session {event.session_id}")This is the answer to incident c41aa9e in the failure log: the agent tried to use a code it read. With a honeytoken, that attempt is the alarm—no user has to notice first.
11. Cost Model
| Item | Cost |
|---|---|
| m8g.xlarge desktop | ~$118/mo |
| gp3 100GB | ~$8/mo |
| NAT + egress | ~$45/mo |
| Ledger + GuardDuty | ~$15/mo |
| Infra total | ~$186/mo |
| Tokens, ~0.5M/task | ~$1.50–4/task |
| × 60 tasks/mo | ~$90–240/mo |
The EC2 line is the cheap part. The loop is the bill.
Always-on is the price of proactivity. The scheduler that makes the agent call you first costs a desktop idling 24/7.
12. Failure Analysis
| Commit | Incident | Root Cause | Control |
|---|---|---|---|
| a3f91c2 | JS-heavy checkout: 14 attempts, CAPTCHA won | T4: bot detection | Accept as unsolved; prefer official APIs |
| 7d02b11 | Approval token expired while user slept; task denied | Gate timeout | Silence resolves to Denied |
| c41aa9e | Agent read 2FA code and tried to use it | T2: prompt injection | Capability scope; honeytoken armed |
| e8802d4 | 0.5M tokens on one train booking | Loop cost | Context compression |
| b17f0a3 | Wrong train, twice | Irreversible action | Rollback snapshot |
| d4f1aa0 | Compressed summary dropped “window seat” | Summarization loss | Separate constraint ledger, never summarized |
13. Conclusion
The $10B valuation is not the model. Everyone has the model. The moat is the accumulated session, the memory that survives disconnect, and a loop cheap enough to run forever.
Securing that is a workload discipline problem. We are good at workload discipline. We have been applying it to the wrong layer: the wrapper, not the loop.
The fix was never “better vibes for the model.” It is:
- Contain the loop.
- Scope the capability.
- Checkpoint the disk.
- Arm the tripwire.
- Make silence mean no.
It costs about as much as the gym. And it will not email your accountant at 2pm.
End of paper.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article