
Agents for Humans: my agent refuses to file the biggest discrepancy it finds
The most expensive finding in my dataset is 3,000 KRW and the agent will not file it. It files a 380 KRW one without asking. Here is why, and what happened when I removed the gate to check it was doing anything.
The most expensive finding in my dataset is 3,000 KRW. My agent will not file it. It files a 380 KRW one instead, without asking.
That looks backwards, and every time I show it to someone they assume it is a bug. It is the single most deliberate decision in the project. This post is about why, and about what happened when I took the enforcement out to check it was doing anything.
Three findings, three different answers
Settlement Sentinel reconciles delivery-platform statements for a small restaurant. Across the three platforms in the demo it finds exactly three problems:
Platform Order Evidence Confidence At stake ActionCoupang Eats CE-2004 rate_mismatch 0.95 380 KRW filed automaticallyBaemin BM-1005 arithmetic_error 0.90 1,000 KRW filed automaticallyYogiyo YG-3005 unexplained_settlement_gap 0.55 3,000 KRW refuses — asks the owner
Read the last two columns together. Confidence goes down as the money goes up. The agent acts on the cheap certain thing and stops on the expensive uncertain one.
That ordering is not a coincidence in the data. It is what uncertainty usually looks like. The problems you can prove are the boring ones — a rate that contradicts the contract, a line that contradicts itself. The big weird gap with no stated cause is big and weird precisely because nobody has explained it yet.
Why each one gets the confidence it gets
Confidence here is not a model output and not a guess about size. It is a fixed property of the kind of evidence, assigned in Python:
if stated_rate != contract_rate: evidence, confidence = "rate_mismatch", 0.95elif commission_delta and settlement_delta: evidence, confidence = "arithmetic_error", 0.90else: evidence, confidence = "unexplained_settlement_gap", 0.55
CE-2004 gets 0.95 because the statement contradicts a document I hold: the contract says 9.9%, the statement charged 10.9%. Two independent sources disagree. There is nothing left to interpret.
BM-1005 gets 0.90 because the statement contradicts itself: the commission is overstated by exactly the amount the settlement is understated by. I do not need any outside document to know something is wrong. It is slightly below the rate case only because the direction of the error is inferred rather than stated.
YG-3005 gets 0.55 because I know the amount and nothing else. The commission is correct. The order total is correct. 3,000 KRW is missing and the statement offers no reason. That is consistent with a platform error. It is equally consistent with a customer refund the owner processed and forgot, a promotional co-pay, or a chargeback. All three are ordinary. Only one of them is worth disputing.
Filing a dispute on that is not a small mistake. The owner sells on this platform every day. Accusing them of a 3,000 KRW error that turns out to be the owner's own refund is worse for the relationship than quietly eating it.
The gate
The decision is nine lines and lives in
gate.py . Nothing else in the codebase is allowed to make it:CONFIDENCE_THRESHOLD = 0.85
if conf >= CONFIDENCE_THRESHOLD: _append(LEDGER_PATH, {**base, "status": "dispute_filed", "decided_by": "agent"}) verdicts.append({**base, "action": "auto_dispute"})else: question = _owner_question(f.get("amount_at_stake") or 0) _append(ESCALATION_PATH, {**base, "status": "pending_owner", "question_for_owner": question}) verdicts.append({**base, "action": "escalated", "question_for_owner": question})
Two things about this are worth more than the threshold value itself.
First,
amount_at_stake is not in the condition. It is carried through the whole pipeline, printed in the report, and totalled at the bottom — and it never touches the branch. Money is something the owner needs to see. It is not an input to whether the agent may act.Second, the else branch is not a fallback. It is the default. "Below threshold" and "missing", "malformed", "unrecognised evidence type" all land in the same place: a question in the owner's queue and nothing in the ledger. Fail-closed means the boring failure and the exotic one produce the same safe outcome, which matters because I can only anticipate the boring one.
What the refusal looks like
Refusing is not the same as doing nothing. Doing nothing is what happens today, and it is why the money disappears. The agent hands the owner exactly one decision, framed so it can be answered from memory in five seconds:
### (c) Questions for the owner- YG-3005 — 3,000 KRW gap, cause unknown. Do you know of a refund or adjustment on this order? Reply 'approve' to file a dispute or 'dismiss' to close.
The owner is the missing information source. They know whether they issued a refund on that order. No amount of statement parsing will recover that, so the correct move is to ask the one entity that has it.
When they answer, the outcome is recorded with
decided_by: owner rather than decided_by: agent , and re-running the platform returns skipped_already_filed rather than filing again. The distinction matters if anyone ever audits the audit: every entry in the ledger says who authorized it.Checking the gate does anything
Here is the part I did not expect.
A threshold is easy to write and easy to believe in. So I ran the pipeline with the gate step skipped — the flagging tool still ran, the model still saw the flagged rows, but
decide_actions never executed.The model did not fail. It wrote a perfectly presentable report on its own. It identified CE-2004, explained the 10.9% versus 9.9% discrepancy, computed the 380 KRW overcharge correctly, and recommended disputing it. Reading it, you would think the system was working.
Two things were wrong with it.
Nothing was filed. Not the dispute, not an escalation, not a ledger line. The narrative said "recommended action: dispute CE-2004" and no action existed anywhere in the system. Without the gate the agent is a commentator.
And it reported "Confidence: 70%."
There is no 70 anywhere in my code. The rule table has 0.95, 0.90 and 0.55. The model produced a confidence-shaped number because the report format had a slot for one, and it filled the slot. It was not lying; it was completing a pattern. But 70% is below my threshold and above nothing in particular, and if a downstream system had read that number and acted on it, the entire safety property would have been decided by autocomplete.
That is the argument for putting the gate in Python, compressed into one observation. Not that the model is unreliable in some general way — it summarised the situation well. It is that a value which must hold cannot come from something whose job is to produce plausible text.
What I would keep in any agent that acts
The threshold number itself is arbitrary; 0.85 fits my three evidence types and nothing more. What generalises is the shape around it:
Score the evidence, not the consequence. The moment stakes enter the confidence calculation, you have built a system that talks itself into acting on the expensive uncertain case.
Make refusing produce something. A refusal that yields a specific answerable question is a feature. A refusal that yields silence is just a failure with better manners.
Put the branch where narration cannot reach it. Whatever the model says about a row, the row's fate is decided by a comparison in Python.
Delete the enforcement and look. If the system behaves the same without it, it was never enforcing anything. Mine failed loudly, which is the outcome you want from that test.
You can watch the refusal happen without AWS credentials:
python agent.py yogiyo --no-llm Previous post: how the same agent passed a 10.9% overcharge as clean, and the one-line bug behind it.
Built for the AWS Agents for Humans hackathon, Professional Agents track.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article