Why Alarms, Not Just Dashboards
A CloudWatch dashboard tells you what's happening if you're looking at it. A CloudWatch alarm tells you what's happening the moment you're not. That difference is the whole point of monitoring: you shouldn't need to babysit a graph to know your infrastructure is in trouble.
In this short walkthrough, I'll show a real CPU spike on an EC2 Auto Scaling instance, from the moment CloudWatch detects it to the moment it resolves — using the same sandbox environment from my previous Terraform article.
1. The Alarm Definition
The alarm watches the CPUUtilization metric on the Auto Scaling group and trips when it crosses 80% for one 5-minute datapoint:
CPUUtilization > 80 for 1 datapoint within 5 minutes
Simple, but effective — this threshold is what separates normal load from a real problem worth waking someone up for.
2. CPU Climbs — the Alarm Triggers
Here's the metric graph as CPU climbs past the threshold, moving from near-idle to over 90% in about 20 minutes:
Once it crosses 80%, the alarm state flips from OK to ALARM , and CloudWatch fires the configured action immediately:
The alarm is wired to an SNS topic, which forwards the event by email. The payload is structured JSON — everything you need to build automated handling on top of it (alarm name, metric, threshold, instance ID):
{
"AlarmName": "CPUHigh-web-01",
"NewStateValue": "ALARM",
"NewStateReason": "Threshold Crossed: 1 datapoint [92.3] was greater than the threshold (80.0).",
"Trigger": {
"MetricName": "CPUUtilization",
"Namespace": "AWS/EC2",
"Threshold": 80.0
}
}
This is the same structure a Lambda function would consume to trigger automated remediation instead of a human reading an email — the logical next step after this basic setup.
4. Back to Normal
A few minutes later, as load drops, CPU falls back under the threshold and the alarm state returns to OK automatically — no manual reset needed:
Key Takeaways
- An alarm is only as good as its threshold and evaluation period — too sensitive, and you get noise; too loose, and you miss real incidents.
- SNS turns a silent metric breach into an actionable notification — email is the simplest integration, but the same topic can trigger Lambda, Slack, or PagerDuty.
- Alarms reset themselves back to
OK automatically once the metric recovers — there's no manual "acknowledge" step required for the state itself.
- The JSON payload CloudWatch sends is structured and predictable, which makes it a natural trigger for automation rather than just a human-read notification.
This CPU alarm is one of the monitoring building blocks for a larger project I'm working on: an agent that reads this exact kind of alarm payload and decides how to react automatically. More on that soon.
Loading article