The question I actually wanted answered
Human-in-the-loop is the default answer whenever someone worries about giving an AI agent purchasing power. Add a confirmation step, and the agent can't do anything reckless. That's the pitch. I'd built wallet-agent , a Strands agent that stops before every purchase and waits for a human to approve it, and I believed the pitch too. What I hadn't measured was the other side of that trade. A confirmation step takes time, and in a scenario where a hundred other buyers are hitting the same limited-stock item at the same instant, time is the one thing you don't have. So I built DROPZERO , a flash-sale storefront on Aurora DSQL that's designed to never oversell no matter how many people hammer it at once, and put the same agent in front of it twice: once with the approval gate on, once with it off. The setup
20 units of one product. Two groups of buyers hitting /api/purchase at the same time, 100 requests each:
- no-approval: fires the purchase the instant it sees stock, standing in for an agent that acts fully autonomously
- with-approval: waits for a simulated human decision before firing, standing in for wallet-agent's approval-card flow
I couldn't run a real approval UI 200 times in one load test, so I modeled "a human looked at the card and decided" as a fixed delay with jitter, and varied that delay from 0ms up to 5000ms:
export function purchaseWithApproval(data) {
const jitter = 1 + (Math.random() - 0.5) * 0.4; // ±20%
sleep((APPROVAL_DELAY_MS * jitter) / 1000);
const res = http.post(`${BASE}/api/purchase`, /* ... */);
tally(res, confirmedWithApproval, soldoutWithApproval, errorsWithApproval);
}
Both groups hit the exact same DSQL-backed inventory row at the same time, so whatever protects against overselling has to hold up under real contention, not a mocked one.
What happened
Approval delay | No-approval (confirmed/soldout/error) | No-approval success | With-approval (confirmed/soldout/error) | With-approval success | Oversold |
|---|
0ms | 13/87/0 | 13.0% | 7/93/0 | 7.0% | none |
500ms | 15/85/0 | 15.0% | 5/95/0 | 5.0% | none |
1000ms | 16/80/4 | 16.0% | 4/96/0 | 4.0% | none |
3000ms | 20/72/8 | 20.0% | 0/100/0 | 0.0% | none |
5000ms | 20/72/8 | 20.0% | 0/100/0 | 0.0% | none |
(The "error" column is OCC conflicts that ran out of retries and came back as a 500. That's a purchase that gave up under contention, not a purchase that oversold — inventory never actually moved for those requests.)
Two things stand out. First, the approval side loses ground fast. Even at 0ms it's already behind (7% vs 13%), just from the extra network round trip, and by 3 seconds it's shut out completely. Every single approved buyer arrived after the no-approval group had already cleared the shelf. Second, and this is the part I actually cared about: oversold count is zero in every single row. Confirmed purchases never exceed 20, with or without the approval step, with or without the delay.
Why the second number matters more than the first
It would be easy to read this as "approval gates are bad, don't use them." That's not what the data says. What it says is that the two things people bundle together under "safety", not overselling and not letting an agent act rashly, are actually being handled by two completely different layers of the system.
DSQL's optimistic concurrency control is what's preventing the oversell. Every purchase attempt reads the current stock, and if another transaction changed it in the meantime, the write gets rejected and retried. That check runs the same way whether the request came from an agent that asked permission first or one that didn't. The approval step never touches that guarantee at all, because it isn't the thing providing it.
So what is the approval step actually buying you? Not correctness, the database already had that. What it costs you is speed, and in a scenario built entirely around speed, that cost can go all the way to zero successful purchases. The value of a human-in-the-loop gate isn't "it stops the system from breaking." It's "it stops a purchase from happening that a human would have said no to", catching a bad call, an unintended amount, a request nobody should have approved in the first place. That's a real job. It's just a different job than the one people usually credit it for.
How it's wired
dropzero-wallet-agent sits between the two existing projects rather than modifying either one. It reuses wallet-agent's local JSON approval store and points its purchase tool at DROPZERO's existing /api/purchase endpoint:
def request_drop_purchase_approval(product_id, product_name, justification):
entry = approvals.request_approval(
resource=f"dropzero:{product_id}",
amount_usd="0",
justification=f"[DROPZERO] {product_name}: {justification}",
)
# ... prints an approval card, waits for `python agent.py approve <id>`
The load test talks to DROPZERO's HTTP API directly rather than driving the agent 200 times, which is what makes the delay-sweep even possible to run.
What's next
This was a single-item, single-region test. Whether the gap closes or widens once you factor in DROPZERO's multi-region writes, or a real Slack approval round trip instead of a modeled delay, is the obvious next thing to measure.
Loading article