The six tests: why one score would lie to you
A single AI health score hides the one failure that matters most. Six separate verdicts keep a strong return from masking a missing kill switch.
Part 6 of the series "Containing the agent, counting the value"
Part A and B were about containment: what you are defending against, the controls that act whether or not the agent cooperates, and how to prove they work. This article opens Part C, where containment stops being a security topic and becomes a line in a management ledger that sits next to cost and value. The bridge between the two halves of this series is a single design decision: measure every production workload six ways, and never blend the six into one number.
That decision sounds modest. It is the most important choice in the whole design, because the alternative — a single AI health score — is the thing that lets a dangerous workload look fine.
The temptation of one number
Every dashboard wants a headline. A single green-amber-red light per workload is easy to build, easy to read, and easy to roll up to a portfolio view a board will nod at. The pull toward one blended score is strong, and it should be resisted.
Here is why. Imagine a workload with an excellent return — it saves real money, the value is graded on hard evidence, Finance has signed it off. It also has a monitor that has never been red-teamed and a stop that has never been drilled. Blend those into one score and you get amber, maybe even green if the financial numbers are strong enough. The one thing a reader most needs to know — that nobody can prove this agent can be stopped — has been averaged into comfort. A composite score does not just lose information. It actively hides the specific failure that matters most, behind the strengths that matter less in that moment.
Gartner makes a related argument on the value side alone: it tells leaders to stop reporting AI value as a single ROI figure and instead track return on investment, return on employee and return on future side by side, because one number cannot carry three different questions. The ledger in this series extends that logic across value and risk. Six questions, six answers, always shown separately.
The six questions
Every production workload is tested six ways. The first three ask whether it is worth running. The last three ask whether it is safe to run.
- Owned. Does every dollar of this workload's AI spend have one accountable owner with the authority to act? Not a committee, not shared IT-and-finance accountability — one named owner, with a sponsor, decision rights, an escalation path and someone who can pause the system. Forrester puts it bluntly for agents: every agent needs its own identity and a named owner, with no unowned autonomy.
- Costed. What does each successful outcome cost, in full, and is that within the agreed ceiling? Full means the complete denominator: tokens including retries, tool calls, evaluation, observability, the share of containment and infrastructure, and the human time to review and correct.
- Proven. Is the claimed value backed by evidence Finance will sign off? Graded evidence, measured against the value case the workload committed to before it reached production. The grading exists because self-report is unreliable: in METR's randomised trial, developers believed AI made them 20% faster when it measured them 19% slower.
- Contained. Does the agent have the containment its risk tier requires, on evidence? This is the evidenced tier from article 5, not the declared one.
- Stoppable. Can we stop the agent in time? A named person who can stop it, a stop drilled end to end, and a measured time to halt.
- Accountable. Can we show a regulator or auditor what happened and who decided? Incident timelines, the OWASP agentic-risk mapping, the Article 73 clock, and an evidence pack.
Articles 2 through 5 were, in effect, a deep tour of tests 4 and 5. The trifecta and autonomy stages feed Contained; the evidenced tiers, exposure time and time to halt feed Contained and Stoppable. Part C now sets those next to the three value tests, which is the first time in this series that risk and value share a page.
Why the split matters
Putting value and risk in the same ledger, scored the same way but never merged, does two things that separate dashboards cannot.
It stops risk being optional. When containment lives only in a security tool and value lives only in a FinOps dashboard, the business case and the kill switch are never read together. The workload with a great return and no tested stop then sails through the review that matters: the one about whether to scale it. Side by side, Stoppable sits next to Proven, and a fail on one is visible against a pass on the other.
It also stops risk swallowing value. The reverse failure is just as real: a security view with no notion of worth treats every workload as pure liability, and the control conversation never has to justify its cost against the value it protects. The ledger carries both, so containment cost is weighed against the value it defends rather than demanded in the abstract.
Five states, and the one that matters most
Each test resolves to one of five states, not two. Pass, attention, fail — and two more that do the real work:
not applicable, and unknown.unknown is the important one. It means an input is missing — no outcome event, no drill result, no evidenced tier — and it is shown in grey and never counts as a pass. This is the opposite of how most dashboards behave, where a missing input quietly reads as green. A workload you have no data on is not a workload that is fine; it is a workload you cannot see. Treating absence of evidence as a distinct, visible state is what stops the ledger flattering itself.And
unknown does not sit grey forever. If the data is still missing after 30 days, it follows the fail path, because "we never measured it" cannot be a way to dodge the review that a fail would trigger. Missing data is a finding, not a blank.A fail escalates; it does not stop
One point matters enough to state plainly, because it is where people assume the ledger does something it does not. A failed test does not halt the workload. The ledger is a measurement layer, not an enforcement one — enforcement stays with the gateway, the policy engine and the runtime controls from articles 3 and 4. What a fail does is open a review with the workload's decision-rights holder, who records a decision — scale, fix or stop — against a due date.
The escalation has teeth proportionate to risk. A fail that persists for two quarters, or a missed decision deadline, goes up to the executive sponsor. A fail on Contained or Stoppable for a high-risk workload goes to the pause authority the same day, because for those two tests on those workloads, the cost of waiting is the incident itself. This is the organisational echo of the series' throughline: the stop has to be owned by someone outside the agent, and so does the decision about a workload that cannot be stopped.
Advisory, then a gate
A measurement system that no one has to act on gets ignored, so the six tests are designed to harden over time. Early on they are advisory — a mirror the organisation holds up to its own estate while it learns what the numbers mean. Later they become the gate: the checklist a workload has to pass to move from pilot to production, and from production to scaled, with exceptions approved by a named sponsor rather than nodded through.
The sequencing is deliberate. Turn a gate on before the measurements are trusted and teams route around it. Introduce the tests as a shared, honest picture first, let the baselines settle, then make them a gate once everyone agrees the picture is fair. A control the organisation believes is fair is a control it will not try to game.
What Part C builds next
The six tests are the shape of the answer. They are not yet a system. Naming the questions does not tell you where the owner's name is stored, how a successful outcome is counted, how an evidenced tier is computed each night, or how any of it reaches a CTO's screen by morning.
That is the subject of the final two articles. Article 7 builds the ledger: the workload register, the telemetry pipeline on top of CloudWatch Omni, the nightly rules engine that turns raw measures into six statuses, and the clean split between what the ledger stores itself and what it only references. Article 8 is about the feedstock — what data each test needs, which of it your platform already emits, which of it nobody emits and you have to generate, and how to start on one workload rather than boiling the ocean. The tests tell you what to measure. The last two articles are how you actually measure it.
References
- Gartner, "Beyond ROI: the three AI business outcomes every leader must understand" (return on investment, employee and future). https://www.gartner.com/en/podcasts/beyond-roi-the-3-ai-business-outcomes-every-leader-must-understand
- Forrester, "The state of agentic AI in 2026" (every agent needs its own identity and a named owner; governance runs alongside the agent). https://www.forrester.com/blogs/the-state-of-agentic-ai-in-2026-companies-are-chasing-few-are-catching/
- METR, "Measuring the impact of early-2025 AI on experienced open-source developer productivity" (believed 20% faster, measured 19% slower). https://arxiv.org/pdf/2507.09089
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article