AWS Builder Center

Builder Center

Agents for Humans: Why Agent Approval Is Not Evidence

Agents for Humans: Why Agent Approval Is Not Evidence

A practical architecture for using an AI agent in release assurance without allowing model output—or human approval—to become unverified truth.

shuo cui

CI systems already tell us that something failed. Large language models can summarize the failure and suggest what to test next. The difficult operational question is different:
What trustworthy evidence is still missing before this exact revision can be released?
I built AssureOps Sentinel for the Agents for Humans Hackathon to explore that question. It is a Strands-powered agent for release assurance, deployed on AWS Lambda. The agent does useful reasoning, but it is deliberately prevented from becoming the authority for execution, evidence, or release readiness.
That separation is the most important design decision in the project.

The tempting design that I rejected

A straightforward CI agent often follows this pattern:
text
failure -> model analysis -> recommended fix -> approval -> execute -> mark resolved
It looks efficient, but several meanings are compressed into one workflow:
  • A model proposal is treated as if it were a verified fact.
  • Approval is treated as proof that the change is safe.
  • A successful command is treated as evidence for every affected requirement.
  • Evidence from one revision can accidentally justify another revision.
  • The same actor may propose, approve, execute, and declare success.
For low-risk automation, some of those shortcuts may be acceptable. They are not acceptable for a system that claims to make an assurance decision.

Four authorities instead of one agent

Sentinel separates the workflow into four authorities:
Concern
Authority
Reasoning and suggested checks
Strands Agent proposal
Authorization to run one action
Independent reviewer
Validity of observations
Typed evidence connectors
Release readiness
Deterministic policy
The end-to-end flow is:
text
synthetic API change + source digest
  -> deterministic impact analysis and proof obligations
  -> Strands Agent proposal
  -> strict schema and exact-scope validation
  -> independent review of one plan digest
  -> one allow-listed action
  -> immutable, exact-revision evidence
  -> deterministic release decision
There is intentionally no direct edge from the model to an approval record, an evidence record, an executor, or the final decision.

What the Strands Agent is allowed to do

The application derives the target revision, impacted assets, and required proof-obligation identifiers before it calls the model. The Strands Agent receives that bounded context and a structured-output contract.
Its response contains concepts such as:
JSON
{
  "proof_obligations": [
    {
      "obligation_id": "obl_contract_delivery_quote",
      "asset": "delivery-quote-api",
      "required_evidence_types": ["contract_test"]
    }
  ],
  "action": {
    "kind": "run_synthetic_contract_suite",
    "obligation_ids": ["obl_contract_delivery_quote"]
  },
  "requires_human_review": true
}
The exact identifiers are server-owned. The response is rejected if it adds an unknown obligation, drops a required obligation, changes the target asset, proposes an undeclared action, or fails the strict schema.
Provider timeouts, malformed JSON, and scope drift do not trigger a permissive path. They produce a deterministic review-only fallback plan. A model failure can never convert  hold  into  ready .

Approval authorizes an action; it does not create truth

The reviewer does not approve an abstract idea. The review decision is bound to:
  • the current plan identifier;
  • the plan revision;
  • a content digest of the plan;
  • the target software revision; and
  • the declared execution boundary.
If the plan changes after review, the approval no longer matches. If the target revision changes, the approval no longer matches. A requesting agent cannot approve its own proposal, and an authenticated user without the reviewer role is denied.
Most importantly, an approved plan still leaves the release on  hold .
In the NimbusCart demonstration, approval opens exactly one synthetic execution path while all four evidence gaps remain visible. The decision changes only after the action emits a JUnit artifact and the evidence connector validates every binding.

Evidence is version-bound

A passing test result is not automatically relevant evidence. Sentinel checks that each observation binds to the required:
  • subject and scope;
  • candidate revision;
  • obligation identifier;
  • source identity;
  • artifact digest;
  • outcome; and
  • verification state.
Missing, stale, rejected, untrusted, unknown, or wrong-scope evidence remains distinct. None of those states is silently converted to passing evidence.
This allows the final policy to remain simple and inspectable: every required obligation must have current, passing, exact-binding evidence. Otherwise the decision is  hold .

Why AWS Lambda was a useful boundary

The public demonstration runs on AWS Lambda behind a Function URL. FastAPI and the React workbench share the same deployment. AWS Secrets Manager supplies the model credential to a least-privilege role, S3 stores a content-addressed deployment package, CloudFormation declares the runtime, and CloudWatch records execution evidence.
The Lambda Function URL is public, but all stateful APIs require a shared synthetic-demo token. Only the health endpoint is anonymous. The model credential is retrieved from Secrets Manager and hydrated in memory; it is not present in source, the deployment template, screenshots, or the public submission.
The hosted demo intentionally uses synthetic data and ephemeral storage. I do not claim that Lambda  /tmp  and SQLite are production durability. PostgreSQL and object storage are the next adapters for a production version.

The result

The live demonstration follows one breaking API change through four impacted assets and four proof obligations:
  1. Strands proposes the bounded plan.
  1. An independent reviewer authorizes the exact plan digest.
  1. The decision remains  hold  with four open gaps.
  1. One allow-listed synthetic suite produces a JUnit artifact.
  1. Four exact-revision evidence bindings are verified.
  1. Deterministic policy derives  ready .
  1. Twelve state transitions remain in a content-hashed audit chain.
This design uses the agent where it is strongest: interpreting context, organizing proof work, and explaining why a human decision is needed. It uses deterministic software where ambiguity would be dangerous.
That is the lesson I will carry into future agent systems:
Agents should propose. Humans should authorize. Evidence should decide.

Try the project

# strands-agents# agentic-ai# aws-lambda# testing# ai-agents
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article