An agent that refuses to guess — what Agents for Humans taught us about auditing our own tool

The tool we already had answered on 0.77% of the day

What "refuses to guess" means in code

1. It measures its own evidence before answering — coverage (what fraction of the day did we actually see), market count, records per market.
2. When the evidence is thin, it says exactly why. Not "insufficient data" — it names the ratio, the markets that fell short, and what would fix it.
3. It normalizes units. Auction prices come per package (4 kg, 5 kg, 10 kg boxes). Averaging raw prices compares a small box to a big one — that flipped "which market pays most" in 9 of 16 date-by-product pairs we measured.
4. It reports the median and flags outliers. One record at 500,005 won/kg among 656 moved a market's mean by 21%.
5. The verdict is computed in code, not asked of the model. A prompt that says "be careful" is a hope. A tool that returns insufficient is a constraint — and the numbers below say how much still got past it.

Three runs, three different answers

• Peaches, Aug 28 — which market? Wonju, 4,183 won/kg, from 184 records across 32 markets.
• Cabbage, Aug 30 — which market? Cannot answer. Zero cabbage auction records nationwide that day.
• Onions, Aug 28 — which market? Cannot answer. Not in this sample, which is not the same as not existing.

What we had to fix to make the refusal hold

• The refusal screen carries no prices. We first showed them labelled "reference only, do not cite". The model cited them anyway in 4 of 6 runs. Labels are requests; removing the numbers is a constraint. After: 1 of 6.
• A tool failure becomes a refusal, never silence. When the tool crashed on an empty day, the model filled the gap with markets that do not exist.
• The answer line is written by code. Letting the model phrase it produced stray Vietnamese and Han characters in about 2 of 6 runs. The shipped interface prints the tool's line: 12 of 12 demo runs, no foreign script. The garbling was not fixed — it is not displayed. We keep both numbers.
• The copied line carries no statistics. A headline that read "only 34.97% (45,301/129,536) of the day" got spun into a table of per-date counts the model never queried. Plain words now; 0 of 6 runs invented a number after the change.
• The model cannot quietly change the question. It once passed a garbled product name into the tool and got a perfectly formatted refusal about a word the user never typed. The agent now compares the product it passed against what the user wrote.

Things we did wrong along the way, kept in git rather than hidden

• We reported "median attempts is still 1, so the retry change didn't slow it down." True of attempts. Wall-clock for the three-scene demo ran 92 s once and 209 s the next time (n=2). Answering a wall-clock question with an attempt count nearly planned a video around the faster figure.
• A teammate's demo answered where ours refused — their machine had a full day cached, ours had the bundled sample. Both were correct; the report was not. The agent now pins itself to the bundled file.
• Our publish-safety checker exempted itself and reported a clean run — right up until the thing it exempted was the thing that leaked. A check that exempts itself is not a check.

Limits we are not hiding

• n = 16 date-by-product pairs, n = 6 model runs. We say "often", not percentages-as-rates.
• Full-day caches cover 2 days. Products were chosen by us. The market-bias result is one day.
• Sufficiency thresholds (at least 2 markets, at least 3 records each) are chosen, not derived.
• Model probes are a local 3B model running on Ollama on one machine. The model provider is pluggable in Strands, and Strands defaults to Bedrock when you do not name one — we always name one explicitly. We did not deploy this to Bedrock or AgentCore during the hackathon window, so we have no measurements on that path and make no claim about it.

Try it