AWS Builder Center

Builder Center

A Rain Alert Agent That Is Not Allowed to Decide

A Rain Alert Agent That Is Not Allowed to Decide

I built a rain early-warning system for my town where the AI agent writes the alert but never decides it: how it is fenced in, what broke, what is missing.

Series: Ferreñafe Rain Alert (1 article)

  1. 1
    A Rain Alert Agent That Is Not Allowed to Decide This article

Sergio Mondragón Silva

Introduction

How much power should you give an AI agent when a wrong sentence can hurt the people who read it?
I live in Ferreñafe, a town on the north coast of Peru, where heavy rain is a known hazard: there is an official  hazard map of the city  (INDECI and UNDP, 2003). The official warnings from SENAMHI (the national weather service) cover a whole region, highlands included. A neighbour reading "orange warning for the coast and sierra" has no way to tell whether that means their street. The information exists. It is just not addressed to anyone. For scale: according to the INDECI statistics compendium (I cited it in an earlier write-up and did not re-fetch it for this post), all six districts of Ferreñafe were under a state of emergency from 4 February to 4 April 2017, during the coastal El Niño.
So I built Ferreñafe Rain Alert: a system that reads the official warning and a forecast for the town's own coordinates, decides a risk level, and writes a short civil-protection message in Spanish.
This is Part 1 of a series. It is about the design decision that came before any agent code: the agent helps only where it cannot mislead. Everything started deterministic and careful, and AI capabilities get added one at a time, only if they add value without weakening the protection against telling citizens something untrue. That is the whole idea of responsible AI in this project, and the rest of the post is evidence for it, including the parts that broke. Part 2 covers the agent's first tool.
This project started as my entry to the AWS Builder Center hackathon  Zero to Shipped ;  the original write-up is here . This series is the longer story.

Step 1: Who Decides?

The tempting design is an agent with tools: one that fetches the warning, reasons about the forecast, decides whether to alert, and writes the message. It is also the one I refused.
If the model decides whether to alert, then a model error becomes a public statement. A hallucinated "evacuate now" and a silent miss are both failures I cannot take back once a person has read them. So the rule I wrote at the top of the project instructions is this:
Fail toward sending the deterministic template, never toward sending something unverified.
In practice the flow is plain code, and the agent is one node inside it:
Plain text
EventBridge Scheduler (6-hourly, America/Lima)
  -> Lambda (RunAlertCycle)
      -> SENAMHI (through an S3 relay) + Open-Meteo     two independent signals
      -> deterministic risk rules                        pure domain, no I/O
      -> AlertPolicy: already alerted for this window?   ends here if yes
      -> compose  <- the composer agent, only if a new alert is authorized
      -> validate, then notify and record
I checked this in the code and not only in my diagram:  RunAlertCycle  calls the composer only after  AlertPolicy  has authorized the send, and the composer's return value never feeds back into a decision. The risk evaluator has six ordered branches and no AWS, HTTP, HTML or model imports at all.


Architecture diagram: scheduler, Lambda cycle, deterministic risk rules, dedup, composer agent, vali



Step 2: What the Agent Is Allowed to Do

The composer agent is a Strands agent running on Amazon Bedrock AgentCore Runtime, using Claude Haiku 4.5 at a low temperature (0.2). Its job is narrow: turn structured data, already decided and already sanitized, into a message a frightened person can act on.
In the version this post describes, tagged   baseline-no-tools  , it has no tools. I wrote   tools=[]  explicitly in  agent/app.py  , with a comment explaining why: "this agent reaches no side effect" is the property that made the whole capability acceptable, and a property that depends on a library default is one nobody notices changing.
But the interesting part is not what goes into the agent. It is what stands between its output and a resident. I think of it as three layers:
Layer
What it does
Where it lives
Fixed rules in the prompt
Ask the model to stay on the operator's checklist, write numbers in digits, treat scraped text as data
The agent's system prompt
The validator
Thirteen rules, evaluated in full, no short-circuit
 domain/message_validation.py 
The deterministic template
The message that goes out when anything above objects
 domain/template.py 
The prompt is a request. A model can ignore a request, and I wrote that sentence into the agent's own docstring so nobody (including me in three months) mistakes the prompt for the control. What actually holds is the validator. The sharpest rule is that every number in the message has to trace back to a value in the request. If the model writes a millimetre figure nobody gave it, the draft is rejected and the template goes out instead.
Two design details I am fond of:
  • The validator returns violations, not a boolean. An operator notice that says "rejected" is useless; one that says  city_missing  and  unknown_number  can be acted on.
  • Invariant V0. The deterministic template's own output must pass every rule. If it does not, the validator is wrong, not the template, because otherwise the fallback would be rejected by the rule that guards the agent.
The direction of error is chosen on purpose. Wrong-strict costs one template message and one operator notice. Wrong-lax puts an invented rainfall figure in front of a community.

Real-World Engineering: When Things Break (and How to Fix Them)

The common thread of what follows is a sentence that is now the first rule in my contributor guide: several defects survived review precisely because a check looked like it had run and had not.

Issue 1: A Cyrillic letter walked past my attribution rule

SENAMHI's terms of use mean the system may only attribute something to SENAMHI if it is either a fact the domain produced or an unaltered quotation of the warning title. I turned that into a validator rule and gave it tests. It was green. Nothing wrong ever reached a resident: this was found in review, before the rule met a real message.
Then a review pass found four ways through. The best one: the mention detector was an ASCII regex,  \bsenamhi\b . Replace one letter with a Cyrillic А (U+0410), which looks identical, and the rule never fires:
Plain text
"El SENАMHI informa que deben evacuar de inmediato."   ->  0 violations
The Root Cause: the rule was skipped, and a skipped rule looks exactly like a passing one. The same family had three more members: a tokenizer that returned nothing for fullwidth letters so that  all([])  passed vacuously, and attribution severed by a newline or by a blank line.
The hypothesis test: I reproduced all four before touching code. For the fix I checked that neither normalization alone was enough: NFKC folds fullwidth Latin but not Cyrillic, and a script restriction catches Cyrillic but not fullwidth, since fullwidth Latin is Latin script. The fix needed both. The attribution rule now has 16 tests across three classes, and the findings log records eleven attack variants contained (that run is from my log; I did not repeat it for this post).

Issue 2: The validator rejected a perfect message, and that was the dangerous failure

In one session, the first live invocation of the agent failed three times, and each fix revealed the next problem hiding behind it. A daily token quota in the first region I tried. Then a parameter pair ( temperature  and  top_p  together) that the SDK accepts at construction and the service refuses at the first call. Those two shouted: an exception, a 500.
The third returned a 200 and a good Spanish message. My own validator rejected it:
Plain text
UNKNOWN_NUMBER: '3' is stated in '%' and traces to no such value on the request
The 3 was a day of the month ("el 3 de septiembre"), and the model had even converted the timezone correctly.
The Root Cause: my number scan compared against numeric fields only, not against the components of the dates the request already carried.
Why it was the dangerous one: a false rejection looks exactly like the system working. The validator objects, the template goes out, the community is served, and the agent could be discarded on every single run while every cycle reports success. At that point, 947 tests were green (the count in my log at that time), and by my own log's account none of the validator's tests had been given a real model's output; they had only seen text I wrote, and I write text that conforms to my own rules.
I did not loosen the rule. The  3  does trace to the request, via the date. I widened what counts as provenance, not what escapes the check, and those two fixes look alike from a distance and are not the same thing.

Issue 3: The prompt is not the control, but it moves the numbers

Before I show the numbers, here is what the agent actually writes. When I first switched the composer on (a dry run against the production runtime, fixtures that clear a threshold, nothing transmitted), the deterministic template and the agent produced these for the same input. The template:
Plain text
Nivel: riesgo inminente
Ventana: del 04/09/2026 00:00 al 07/09/2026 00:00 (hora local, America/Lima)
Motivos:
- Aviso oficial del SENAMHI, nivel naranja: PRECIPITACIONES EN LA COSTA NORTE Y SIERRA.
The agent:
Plain text
Alerta inminente en Ferreñafe por precipitaciones en la costa norte y sierra.

Motivos:
Advertencia oficial de precipitaciones en la costa norte y sierra.
It kept all five recommendations verbatim, but it dropped the time window, dropped the source (no SENAMHI, no colour) and renamed the level. The validator accepted it, because nothing in those thirteen rules requires the window or the source. I looked at both side by side and accepted the agent's wording. That was a mistake: in an alert system, a message without its source and its time window is a weaker message, and I let it through because the validator said yes. The system prompt I added later asks the agent to keep the window and to mention SENAMHI only in approved ways, which mitigates it. But a prompt is a request, as I said above, and the validator still has no rule requiring either. That rule is still pending. I include this because it is the honest picture: the agent is useful for phrasing, and the validator guarantees what it guarantees, no more.
Later, a live acceptance script (10 runs, minimum 8 accepted) measured the same agent with and without a system prompt, on the same day:
  • Without it: 6 of 10 drafts accepted. All four rejections were  city_missing : the body said "en la zona", and Ferreñafe appeared only in the title.
  • With it: 10 of 10 accepted.
The caveat is loud: n=10 per configuration, one scenario. It does not say the prompt makes the agent safe. It says the prompt reduces how often the safety net has to catch it, which is a quality-of-service matter, while the validator stays the thing that decides.
The first real warning the system saw (SENAMHI No. 392, orange, valid 2 to 4 October 2026) is a separate episode, and it came before the system prompt shipped. The agent's draft was rejected with  city_missing  and an unknown number ( '2' traces to no value on the request ), and the deterministic template went out. My evidence file records the operator notice but not the rejected draft's text, so I cannot show it to you. What I can say is that the same failure class, a missing city name, is exactly what the later prompt fixed, and that the rule worked on its first real input.

Issue 4: The alarm I built and then deleted

A pre-deploy security review found one risk with no mitigation: a single forged row in the dedup table would suppress alerts for the whole lookback window (72 hours, per the port's documentation) while every alarm stayed green. Narrowing IAM does not help, because the attacker writes into the partition the function legitimately owns, so the control had to be detection. I built a CloudWatch metric filter for "an elevated level that did not send", tested it, and sent it to review.
The Root Cause (of the alarm's failure): that pattern is structurally identical to a normal day of sustained rain, where the system correctly does not re-alert for an event it already announced. The filter matched the attack and the weather.
An alarm that fires on ordinary rain gets muted, and a muted alarm reads as coverage. So I deleted it, and wrote the finding onto the  Notifier  port, the one place the next person will be standing when the risk becomes real. Which brings me to what is missing.

Step 3: What It Does Today

What is deployed and running: the scheduled Lambda (the switch is  scheduleEnabled  in the CDK config, a reviewed one-line change instead of a flag someone can forget), the DynamoDB table and SSM parameters, CloudWatch alarms (up to four; the relay alarm exists only while the relay is enabled, which it is), a public status page on AWS Amplify, the AgentCore runtime, and a relay for SENAMHI data. The relay exists because the deployed cycle could not reach SENAMHI from AWS; a machine outside AWS fetches the page and pushes a validated copy to a private S3 object. That story deserves its own post.
The public page is deliberately honest about what it does not know: if data is stale, the age goes first, above the risk level; if a source was unavailable, a notice under the heading names which one. A quiet failure that looks like good news is the worst failure for a page like this.

What's Still Missing

The most important gap first, and I want it to stay uncomfortable:
  • Nothing sends an alert to a person.  ConsoleNotifier  is the only notifier in the codebase and it says so in its own output: nothing was transmitted. That is structural, not a flag. Real recipients are personal data, and consent matters. Today the public page is the channel, so a real orange warning reached nobody's phone.
  • The obvious channel does not fit. When I read Meta's WhatsApp Groups documentation, it required an Official Business Account, joining was by invite link only, and the documented limit was 8 participants per group. That is a work crew, not a town.
  • The relay depends on my laptop. An always-on producer is the next job, before the rainy season.
  • The validator does not require the window or the source. Issue 3 showed the gap; today only the prompt covers it.
  • No managed guardrails yet. The validator is hand-written domain code. I want to evaluate managed guardrails and other mechanisms as additional layers, never as a replacement for the validator. Nothing like that exists in the repository today.
  • Delivery changes the threat model. The forged-record risk from Issue 4 is survivable only because nothing delivers. The first notifier that actually sends has to resolve it first, and the port says so.

How I Built It (the Short Version)

I used Claude Code throughout, with spec-driven development: a spec and design per change, then tasks, then strict test-first implementation with hand-written fakes instead of mocks. Every change gets a security review on the branch diff before a pull request, and several reviews were done by a fresh agent with no memory of writing the code. A findings log in the repository records each surprise next to the command that revealed it. That process produced most of the findings above. It is not the point of this series; the agent is. But it is why I could attack my own validator and find holes.
For the AWS side I connected Claude Code through the  Agent Toolkit for AWS : one  aws configure agent-toolkit  installed the AWS MCP Server and a set of AWS-authored skills, authenticated with short-lived  aws login  credentials instead of a static access key. The pre-deploy checks followed the toolkit's AgentCore deploy skill, and the region choice was checked against current regional availability.

Practical Tips

  • Decide what the model may never decide. Write it down before writing the agent, and make the code enforce it, not the prompt.
  • Make the safe path the default. A deterministic fallback that is tested against the same rules as the agent keeps a "no" from becoming an outage.
  • Return reasons, not booleans. Operators fix what they can read.
  • Test with real model output. Everything I imagined passed; the first real output did not.
  • Delete detections that cry wolf. An alarm that fires on normal operation costs attention and manufactures false confidence.

Conclusions

  • Fence first, capability second: the agent writes the words; code decides whether anyone gets them, and thirteen rules plus a template stand between its draft and a resident.
  • A skipped check looks like a passed check: the Cyrillic bypass, the empty token list and the silent false rejection are the same bug in different clothes.
  • The prompt moves quality, the validator holds safety: 6/10 to 10/10 acceptance changed how often the net was used, not whether it was needed.
  • The honest caveat: my evidence is a handful of live runs and one real warning. It shows the design works as intended; it is not a measure of how the agent behaves over a rainy season.

Next in the Series

So what does it take to let this agent see something, safely? In Part 2 the agent gets its first tool, a read-only  get_recent_alerts , and I explain why that one was safe to hand over when almost any other tool would not have been.

References

  • INDECI and UNDP, Mapa de Peligros de la ciudad de Ferreñafe (2003)
  • INDECI, Compendio Estadístico 2017, Gestión Reactiva (state of emergency table)
  • SENAMHI official warnings; Open-Meteo forecast API
  • Amazon Bedrock AgentCore Runtime documentation; Strands Agents documentation
  • Meta, WhatsApp Groups API documentation
P.S. The only thing in this project that has been confidently wrong is my own regex. The model, at least, has the decency to be rejected by it.
# agentic-ai# social-good# social-impact

Series: Ferreñafe Rain Alert (1 article)

  1. 1
    A Rain Alert Agent That Is Not Allowed to Decide This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article