AWS Builder Center

Agents for Humans: interventions are a control plane, not a prompt

Why an anti-hallucination gate whose predicate is decidable beats one written into a prompt, and where in the Strands loop it has to sit to survive a session restore.

I am building Second Budget for the AWS "Agents for Humans" hackathon, Good Neighbor Agents track. It sits with a household holding a SNAP notice of action and answers the only question they have: is this number right? It re-derives the allotment from the underlying facts, and when it disagrees with the notice, it says what would have to be true about the household for the notice to be correct.
The output is a draft fair-hearing request. Every number in it is checkable, and someone will check it: an agency representative at a hearing looks the cited paragraph up and reads it back. So the sentence "the model must not state a number it did not compute" has to hold — not usually, not in eval, always.
You can write that sentence in a system prompt. It will eventually be violated, and you will not know when. The alternative is to make it a gate with a decidable predicate, positioned where a denial actually takes effect. That positioning turned out to be the hard part, and most of what follows is what I measured about it.

A fake gate and two real ones

Here is the gate that looks like compliance and is not:
if not payload.get("citation"):
return Deny(reason="citations are required")
It is a non-empty check. It admits  citation: "7 CFR 273.9"  attached to a paragraph that says something else entirely, and it admits  citation: "x" . The predicate is decidable but it decides the wrong thing. The test for a real gate is whether a wrong answer can pass it, not whether a missing answer can.
NumbersGate. When the engine node finishes a budget it freezes a certificate: the value of every stage it computed, plus the figure printed on the notice, because the packet has to be able to state what the agency decided. The gate extracts currency-shaped tokens from the payload of a guarded tool call and requires set membership. That is it — no model in the loop, no judgment, just a set.
The predicate, from  src/second_budget/engine/certificate.py :
_MONEY = re.compile(r"\$?\d{1,3}(?:,\d{3})+(?:\.\d{1,2})?|\$?\d+(?:\.\d{1,2})?")

def money_tokens(text: str) -> tuple[float, ...]:
out = []
for match in _MONEY.finditer(text):
try:
out.append(float(match.group(0).lstrip("$").replace(",", "")))
except ValueError: # pragma: no cover - the pattern makes this unreachable
continue
return tuple(out)

@dataclass(frozen=True)
class Certificate:
values: frozenset[float]
stage_of: dict[float, str]
cfr_of: dict[float, str]

def permits(self, value: float) -> bool:
return round(value, 2) in self.values

def unpermitted(self, text: str) -> tuple[float, ...]:
return tuple(v for v in money_tokens(text) if not self.permits(v))
and the gate itself, from  src/second_budget/control/interventions.py :
from strands import InterventionHandler
from strands.interventions import Deny, InterventionAction, OnError, Proceed

def _payload_text(tool_use: dict[str, Any]) -> str:
"""Everything the model is about to write, flattened for inspection."""
return json.dumps(tool_use.get("input", {}), default=str)

class NumbersGate(InterventionHandler):
name = "numbers-are-not-yours"

def __init__(self, certificate: Certificate, *, guarded_tools: frozenset[str]) -> None:
self.certificate = certificate
self.guarded_tools = guarded_tools
self.denials: list[tuple[str, tuple[float, ...]]] = []

def before_tool_call(self, event: Any, **_: Any) -> InterventionAction:
tool_use = event.tool_use
if tool_use.get("name") not in self.guarded_tools:
return Proceed()

offending = self.certificate.unpermitted(_payload_text(tool_use))
if not offending:
return Proceed()

self.denials.append((tool_use.get("name", ""), offending))
available = ", ".join(
f"{value:g} ({self.certificate.explain(value)})"
for value in sorted(self.certificate.values)
)
return Deny(
reason=(
f"{', '.join(format(v, 'g') for v in offending)} was not computed by the "
f"engine and may not appear in a filing. Use one of these values, or call "
f"the engine again: {available}"
)
)
The reason string is the model's entire steering signal, so it is written as an instruction rather than a complaint. A test scripts a model that writes "Your net income is $1,800.", takes the denial, and writes the engine's allotment instead: three model calls, one denial, one line in the document.
CitationGate. Same shape, different predicate. A quote must be a literal substring of the section it cites, after whitespace normalisation. Not similar to. Not semantically consistent with. A substring.
def before_tool_call(self, event: Any, **_: Any) -> InterventionAction:
tool_use = event.tool_use
if tool_use.get("name") not in self.guarded_tools:
return Proceed()

payload = tool_use.get("input", {})
quote = payload.get(self.quote_field)
if not quote:
return Proceed()

# A lookup that failed earlier makes every later quote unverifiable.
self.store.assert_healthy()

citation = str(payload.get(self.citation_field, "")).strip()
if not citation:
self.denials.append(("", str(quote)[:80]))
return Deny(
reason=(
"a quotation needs the citation it came from. Name the section, "
f"for example '7 CFR 273.9', in the '{self.citation_field}' field."
)
)

source = self.store.section_text(citation)
if source is None:
self.denials.append((citation, str(quote)[:80]))
return Deny(
reason=(
f"{citation!r} is not a section of 7 CFR 273 in the index. "
f"Cite one of: {', '.join(self.store.citations()[:6])}, ..."
)
)

if self._normalise(str(quote)) not in self._normalise(source):
self.denials.append((citation, str(quote)[:80]))
return Deny(
reason=(
f"that text is not a literal span of {citation}; it was "
f"paraphrased. Quote the regulation word for word, or quote a "
f"shorter span that you can reproduce exactly."
)
)

return Proceed()
A word on what the index contains, because it is the same discipline applied one level down. I wanted paragraph-level citations —  7 CFR 273.10(e)(2)(ii)(C) , the level a caseworker actually argues at. eCFR does not publish them. You get a flat list of  <P>  elements and you infer depth from four alternating marker alphabets. Two rounds of heuristics still produced 187 duplicate citations across 2,418 paragraphs, 7.7 percent of paths wrong. A wrong citation is worse than a coarse one, because the representative looks it up and reads it back. So the index keys on the section, which eCFR gives explicitly as the  N  attribute and which therefore cannot be wrong: 32 sections of 7 CFR 273. A  collisions()  function and a test that asserts it is still non-empty keep that measurement reproducible. CitationGate is only as good as the thing it checks against, and I would rather it check against something coarse and true.

Why both of these sit on  before_tool_call 

The obvious placement for a correction is  after_model_call : the model has spoken, you inspect what it said, you correct it. Two things rule it out.
A  Guide  at  after_model_call  does not survive a session restore.  interventions/registry.py:182  appends the guidance straight into  agent.messages . That bypasses  Agent._append_messages  ( agent/agent.py:1706 ) and therefore the  MessageAddedEvent  a session manager subscribes to ( session/session_manager.py:46 ); the SDK documents it at  interventions/actions.py:77-80 . I ran it against a real  FileSessionManager : one invocation, then an append of exactly the shape  registry.py:182  makes, then a fresh  Agent  on the same session id. The model's output came back on restore. The injected guidance did not. A  Deny  at  before_tool_call  becomes a tool result, and the tool-result message goes through  _append_messages  ( event_loop/event_loop.py:898 ), so it persists. If your correction has to still be true tomorrow, it has to be a tool result.
 Deny  at  after_model_call  is a silent no-op.  registry.py:191  logs a warning and continues, and the action-compatibility matrix in  actions.py  marks that cell "—". You cannot block the model's prose. Once you accept that, the architecture writes itself: if prose cannot be gated, then no number may originate in prose. In this build the packet is not written by a model at all.  packet/render.py  emits it and records the origin of every monetary figure as it writes it —  engine: ,  fact: ,  reconciliation: ,  derived:  — and  verify()  re-checks every  engine:  figure against the certificate before the packet is returned, raising rather than shipping a document that reads well and cannot be defended. NumbersGate is the same predicate one layer out, at the tool boundary, for the case where a figure travels through a tool call.
This is a defensible design on Strands' side, not a bug.  after_model_call  fires after the provider has returned; there is no channel left to un-say a token. The SDK is telling you where the seam is. I put the gates on the seam.

 on_error , and the assumption to unlearn

Every intervention handler carries an  on_error  policy for when the handler itself raises. I assumed the default was fail-open. It is not ( interventions/handler.py:32-40 , and  registry.py:246-263  for the dispatch).
  • The default is  "throw" . Your handler's exception is re-raised.
  •  "proceed"  is the one fail-open value — the tool call goes through unchecked.
  •  "deny"  fails closed and keeps the agent alive, because the denial comes back as a legible tool result the model can respond to.
For a compliance gate,  "deny"  is what you want.  "throw"  is honest but kills the run on a regex edge case in a gate that was supposed to be defensive;  "proceed"  turns a broken gate into no gate at all, quietly. It is not a registration option — it is a property on the handler, so both gates override it:
@property
def on_error(self) -> OnError:
# Fail closed, but keep the case alive. See the module docstring.
return "deny"
and the handler itself goes into a plain list:
gate = NumbersGate(certificate, guarded_tools=frozenset({"write_finding"}))
agent = Agent(
model=model,
tools=[write_finding],
interventions=[gate],
callback_handler=None,
)

Guide hazards worth knowing before you reach for one

 Guide  is still the right tool for shaping behaviour that is not a hard requirement. It has three edges, and together they are why this build has no  after_model_call  handler at all.
It has no retry cap —  interventions/actions.py:72-75  says so plainly: "The framework imposes no retry cap on guide-triggered retries." And it also fires on the exception path:  event_loop/event_loop.py:615-637  catches the model exception, builds an  AfterModelCallEvent  with  exception=e  set, dispatches it, and honours  retry  from it. A naive handler that retries whenever it does not see the output it wanted will therefore retry against a dead provider. Anything living there has to check  event.exception  before doing anything else, and count its own retries.
Third: guidance accumulates in the message list, and the default  SlidingWindowConversationManager(window_size=40)  trims from the front. Enough guidance and you have silently deleted the household's actual facts to make room for repeated instructions about formatting.

The fail-open path in memory

The regulation index is a  MemoryStore  behind a  MemoryManager .  memory/memory_manager.py:353  catches a store's search exception, logs a warning and skips that store. I pointed a  MemoryManager  at a single store whose  search  raises: it logged one line and returned  [] . That is a reasonable default for a chat assistant — a flaky index should not take down the conversation. For this build it is the worst possible failure, because a dead index and a regulation that says nothing are the same value:  [] . The agent would conclude the household has no case.
It bit us in a specific way. Every synchronous entry point goes through  run_async , which builds a fresh  ThreadPoolExecutor  each time ( strands/_async.py:31 ), so the store's constructor, its  initialize  and each  search  run on three different threads — I instrumented the store and logged three distinct thread ids across one  Agent(...)  construction and one turn. A single shared  sqlite3  connection therefore raises  ProgrammingError  ("SQLite objects created in a thread can only be used in that same thread"), which  MemoryManager.search  then swallows into "the regulation is silent."
Three seams answer it.  initialize  is the unguarded one —  memory_manager.py:607-611  awaits each store's  initialize()  with nothing wrapped around it — so a startup canary there, searching for a string known to be in the corpus and raising if it does not come back, fails the run closed at  Agent(...)  construction rather than mid-packet. Connections are thread-local, and  check_same_thread  is left on, because the guard is what reported this in the first place. And the store latches its own failures, so  assert_healthy()  can refuse to let anything be quoted after a lookup has failed mid-case — which is why it is the first thing CitationGate calls.
The memory injection format is custom for a separate reason: the default at  memory_manager.py:735  renders  content  and the store's name and discards  metadata , so the section number never reaches the model and CitationGate becomes unsatisfiable by construction.

One human gate for twelve facts

The last intervention is the one aimed at the human. Elicitation produces a fact ledger, and some entries are stated by the household while others are inferred by the model. Confirming those one at a time is how you get a person to click yes twelve times without reading.
 BeforeToolsEvent  fires once per cycle, for the whole batch of tool calls the model returned, which is what makes one gate for twelve facts possible:
def register_hooks(self, registry: HookRegistry, **_: Any) -> None:
registry.add_callback(BeforeToolsEvent, self._confirm_batch)

def _confirm_batch(self, event: BeforeToolsEvent) -> None:
proposed = _fact_calls(event, self.tool_name)
if not proposed:
return

self.batches_presented += 1
payload = {
"facts": [
{
"fact_id": use["input"].get("fact_id"),
"value": use["input"].get("value"),
"provenance": use["input"].get("provenance"),
"source": use["input"].get("source", ""),
"needs_scrutiny": use["input"].get("provenance") == Provenance.INFERRED.value,
}
for use in proposed
],
}

decision = event.interrupt(
name=f"{INTERRUPT_NAME}:{_batch_key(payload)}", reason=payload
)

self.gates_paused += 1
rejected = set(decision.get("rejected", []) if isinstance(decision, dict) else [])
self.rejected.extend(sorted(rejected))
if rejected:
event.cancel = (
"The navigator rejected: " + ", ".join(sorted(rejected))
+ ". Do not record these as stated; ask the household again."
)
The interrupt name has to be derived from the batch's content, and getting it wrong fails in two directions.  BeforeToolsEvent  builds the interrupt id from the name alone — uuid5 over the name,  hooks/events.py:169  — and an interrupt id that already carries a response is returned rather than raised ( types/interrupt.py:107-111 ), so a fixed name lets approval of the first batch silently cover the second. A counter fails the other way: the hook re-runs when the graph resumes, so the same batch would get a fresh id and the loop would never terminate. A hash of the proposed facts has both properties at once.
A test scripts twelve  record_fact  calls in one assistant turn and asserts what came back: one interrupt, twelve facts in its payload, one model call, and an empty ledger until a human answers. Rejecting one cancels the batch with a message naming it, so nothing is recorded and the frontier stays open — after a partial rejection the ledger is still empty, and the graph runs another elicitation round.
What the action layer does not give you is the aggregate. Three  Confirm  actions over a batch of three tool calls arrive in a single interrupt round, but as three separate interrupts, one per tool use, each with its own prompt — I measured it.  Confirm  is scoped to a single call by design, so presenting a batch as a batch, with the inferred rows marked for scrutiny, belongs in the hook, where you can see the whole set at once.
The pattern underneath all of these is the same. Ask what predicate decides the question, check that it is decidable without a model, then find the hook where a denial actually takes effect and persists. In this build that is  before_tool_call  for everything that must hold,  initialize  for the canary, and  BeforeToolsEvent.interrupt  for the one decision a human should make. 163 tests cover them, running in about 1.8 seconds with no AWS credentials in the environment — but that is the subject of another post.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article