AWS Builder Center
Agents for Humans: Excel Has No Unit Tests

Agents for Humans: Excel Has No Unit Tests

Spreadsheets are the most used programming environment on earth, and the only one with no tests, no linter and no CI. I built the missing one with the Strands Agents SDK, on one decision: deterministic analysis finds candidates, agents judge the survivors. Includes the R1C1 trick that makes drift detectable, why tokens scale with anomalies rather than cells, and the day my clean corpus caught three bugs that fired on every well-formed spreadsheet.

Spreadsheets are the most widely used programming environment on earth. They run budgets, forecasts, payroll, inventory, grant reporting and board packs.
They are also the only programming environment with no tests, no linter, no code review and no CI.
I spent this hackathon building the missing one.

The bug you cannot see

Here is the entire problem in one interaction. You have a budget with eight departments and a total underneath. You insert a row for a ninth.
The total doesn't change.
Not because anything crashed — because the SUM still says D2:D9 and your new row is at D10. Nothing turns red. No dialog appears. The total is simply now wrong, and it looks exactly as plausible as it did ten seconds ago.
That class of failure has an unusually grim track record. Public Health England lost roughly 16,000 COVID-19 cases because an XLS sheet hit a row limit and dropped the overflow silently — contact tracing never reached those people. The Reinhart–Rogoff austerity paper's spreadsheet excluded rows from an average, and the conclusion shaped fiscal policy across several countries. JPMorgan's "London Whale" risk model contained a copy-paste error between two cells.
Field audits of production workbooks keep finding defects in the large majority of files examined. Everyone in finance knows this. Nobody checks, because there has never been anything to check with.

Why this can't be an app

My first instinct was a web app: upload a workbook, get a report. I talked myself out of it within an hour, and the reason turned out to be the whole design.
Nobody would ever open it.
The defect is invisible by construction. There is no moment at which a person thinks "I should go and check whether my SUM ranges are still correct." If there were, they would already have checked. An app that waits to be opened will never be opened.
The only form factor that can work is something that watches continuously and speaks once — which is exactly what this hackathon asked for: runs autonomously, surfaces only when there's a real decision to make.
That reframing is what made the rest of the project tractable. The product is not "find problems in spreadsheets." The product is deciding that almost nothing is worth interrupting you for.

The architectural decision everything else follows from

The obvious build is: hand the workbook to a model, ask what's wrong. I think this is a trap, and avoiding it is the most important decision in the project.
A model reading a spreadsheet hallucinates cell references, cannot scale past a few thousand cells, and — worst of all — cannot prove anything. "I think D11 might be wrong" is not something you can put in front of a CFO.
So Plumb splits in two: deterministic analysis finds candidates, and agents judge the survivors.
The first half is exact graph analysis in Python. No model, no credentials, about four milliseconds. It parses every formula, builds a dependency graph, and runs fourteen detectors that produce candidates carrying hard evidence.
The central primitive is A1 to R1C1 normalisation. In ordinary A1 notation, "=B2C2" and "=B3C3" are different strings, so a column of three hundred such cells looks like three hundred unrelated formulas. Normalised to relative notation, both become "=RC[-2]*RC[-1]" — identical. Any cell whose normalised form differs from its neighbours has drifted from the pattern. One transform, and the highest-value detector in the suite falls out of it.

Where the agent earns its place

If the deterministic half is so capable, why have an agent at all? Because there is exactly one question the arithmetic cannot answer, and it is the question that decides everything: is this a defect, or did someone mean it?
A hardcoded number on a sheet of assumptions is how that sheet is supposed to work — someone typed a growth rate in. Correct. The identical hardcoded number on a sheet of computed results means somebody painted over a formula, and the model stopped responding to its own inputs from that day forward. A defect, and an expensive one.
Structurally these are the same thing. Telling them apart needs to know what the sheet is for — and that is judgement, not analysis.
So the Strands Agents SDK layer does exactly that and nothing else. An interpreter agent reads sheet names, headers and labels to work out what kind of sheet it is looking at. Then judge agents run concurrently, one per candidate, each returning structured output rather than prose: a validated verdict object carrying a ruling of defect, deliberate or noise, a severity, a confidence score, a one-line human summary, and a suggested fix.
Structured output mattered more than I expected. Every verdict arrives as a validated object, so nothing downstream is ever parsing prose and hoping. A candidate that cannot be parsed into a verdict is dropped rather than guessed at — the cost of a missed defect is far lower than the cost of a confidently wrong interruption.
The pleasing consequence of this split is that tokens scale with anomalies, not with cells. A hundred-thousand-cell workbook with three problems costs the same as a hundred-cell workbook with three problems.

Then it tells you what it costs

"Possible error in D11" is easy to ignore. So for every finding, Plumb substitutes the corrected formula, re-evaluates the workbook, and reports the difference. It will tell you that D11's SUM covers D2:D7 but the data continues to row 9, that the fix is to widen it to D2:D9, and that this understates the total by 270,734.
That figure is not an estimate or a severity score. It is the amount the number moves. And it propagates: the mistake is in D11, but the thing anyone actually reads is the Summary sheet, and Plumb says so.

The war story

Here is the part I would want to read in someone else's post.
I built two corpora: workbooks with injected defects to measure recall, and matched clean twins — same structure, no defects — to measure false positives. Every defective workbook has a clean twin, so a finding on a twin is unambiguously noise, with no argument to be had about intent.
The clean corpus immediately caught three bugs the recall tests had happily passed. Subtotals were being pulled into the blocks they summarise, so a total sitting directly under its column got compared against the row formulas above it and flagged. The range walker counted a total row as data its own SUM had missed, so every correctly written total looked like it was under-covering. And the outlier detector skipped its single strongest signal: a column of identical values with one outlier has a median absolute deviation of zero, and my guard treated that as a degenerate case to skip rather than a maximal one.
The first two fired on essentially every well-formed spreadsheet in existence. Had I only measured recall, I would have shipped a tool that flagged a false positive on every real workbook it saw, and demonstrated it on stage.
That is when the metric priority inverted for me. The false-positive rate is the headline number, not recall. A watcher that cries wolf gets muted, and a muted agent is a dead agent. A missed defect is survivable; a cried wolf is not.
The current numbers, over 400 matched pairs and reproducible from a seed: recall of 396 out of 400 (99.0%), a single false positive across 17,748 clean cells (0.006%), and a median scan of about 4 milliseconds.
There is a detail in there I am oddly proud of. Recall used to read 99.8%. It went down when I added detectors — not because anything regressed, but because the larger corpus finally gave the weakest detector enough samples to reveal its true rate. I published the lower number and explained why, rather than quietly keeping the better one. Every other detector is at 100%.

What it looks like now

It watches a file and stays quiet. When something breaks, it says one thing, with the amount attached, and — because it keeps a history of every run — it can tell you when: "The Q3 total stopped covering the last two departments on Tuesday. It is understated by 270,734."
That points at a specific edit in a specific meeting. "The total is inconsistent" points at nothing.
It is open source, and there is a live demo you can drop your own workbook into — it runs entirely in your browser, so the file never leaves your machine. I will write about how that ended up being possible, and about teaching an agent when to stay quiet, in the next posts.
Built with the Strands Agents SDK for the AWS Agents for Humans hackathon.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article