AWS Builder Center

Builder Center

AWS WAF Best Practices, Part 2: Bot Control, Anti-DDoS, Logging to SIEM, Monitoring, Fine-Tuning and Cost

AWS WAF Best Practices, Part 2: Bot Control, Anti-DDoS, Logging to SIEM, Monitoring, Fine-Tuning and Cost

Running AWS WAF in production: Bot Control Common vs Targeted, the Anti-DDoS rule group, CAPTCHA and Challenge, logging to S3 or Firehose to your SIEM, dashboards and alarms, the 14-day Count-mode rollout, label-based exceptions and a worked cost example.

DIPNARAYAN

Security Engineer, AWS Certified Security Specialty and Solutions Architect
By Dipnarayan Nandi, Cloud Security Engineer
In  Part 1  we built the rules: a web ACL on CloudFront and the ALB, trust lists and business rules first, then IP reputation, the OWASP baseline and the managed rule groups for your stack, each with its real name, its use case and its cost.
A web ACL full of good rules still fails in production for three reasons: it can't tell a bot from a customer, nobody can see what it's doing, and it was switched to Block on day one and broke something. This part fixes all three.
Here's the path:
  • Bot Control (Common vs Targeted) and AI traffic
  • Layer 7 DDoS with the Anti-DDoS rule group
  • CAPTCHA, Challenge and custom responses
  • Logging to S3, CloudWatch Logs or Firehose → SIEM
  • Monitoring: metrics, alarms, console views and SOC detections
  • Fine-tuning: 14 days in Count mode, then enforce
  • What it all costs, with a worked example


Bots, DDoS and challenges

Bot Control (Common vs Targeted) and AI traffic

 AWSManagedRulesBotControlRuleSet  comes in two inspection levels:

Common
Targeted
What it catches
Bots that identify themselves: search engines, scrapers, HTTP libraries, automated browsers, monitoring tools
Everything in Common, plus sophisticated bots that pretend to be humans
How it detects
Static request analysis: user agent, bot signatures, verification of known bots by origin IP
Adds browser interrogation and fingerprinting (via the WAF token), behavioural and volumetric analysis per session, token-reuse detection across IPs, countries and ASNs, and machine learning for coordinated bot activity
Needs the WAF token?
No
Yes, for most rules. Tokens come from the Challenge/CAPTCHA actions or the AWS WAF JavaScript and mobile SDKs
Typical actions
Block unverified bots by category. Verified bots such as Googlebot are labelled, not blocked
CAPTCHA, Challenge and Block, depending on confidence
Warm-up
Immediate
Volumetric rules about 5 minutes. ML baselines up to 24 hours
Price (beyond standard WAF)
$10/month per web ACL + $1 per million requests after the first 10M free each month
$10/month per web ACL + $10 per million requests after the first 1M free each month
Best for
Most websites and APIs
Login, checkout, ticketing, sign-up, pricing pages: anywhere bots cost you money
How I deploy it:
  • Scope it down. Don't run Bot Control on static assets ( /static/ ,  .css ,  .js , images). You'll pay per request for traffic that doesn't matter. A scope-down statement on the rule group fixes this.
  • Common everywhere, Targeted only on high-value paths. Use two Bot Control rule group references with different scope-downs if needed.
  • Verified bots are labelled, not blocked. If you want to block some verified bots, write a label-match rule after the group.
  • Start in Count. Several low-confidence Targeted rules (such as  TGT_TokenReuseIpLow  and  TGT_ML_CoordinatedActivityLow ) can produce false positives.

AI bots and agents

AI crawlers and agents are now a real share of traffic. Two things help here:
  • The AWS WAF AI activity dashboard (launched February 2026) shows AI bot and agent traffic across your resources: the most active bots, the paths they hit, and verified vs unverified. It's included at no extra cost.
  • Bot Control's  CategoryAI  rule lets you decide what AI crawlers can do. It applies to verified and unverified AI bots, so you might allow AI search crawlers on your public docs while blocking them from pricing or customer data. On CloudFront, AWS WAF can even respond to AI bots with HTTP 402 Payment Required and charge them for content.


Layer 7 DDoS protection with the Anti-DDoS rule group

AWS Shield Standard (free) protects against network-layer (L3/L4) floods. HTTP floods are layer 7 and need WAF. The  AWSManagedRulesAntiDDoSRuleSet  (50 WCU) is built for exactly this.
How it works:
  1. It learns your resource's traffic baseline.
  1. When traffic deviates significantly, the resource is in an event and every request gets the  event-detected  label.
  1. Requests from suspected attack sources get  ddos-request  plus a confidence label: low, medium or high suspicion.
  1. It mitigates with soft actions (silent browser Challenge) and hard actions (Block), each tunable to low, medium or high sensitivity.
Its three rules:
Rule
Default
What it does
ChallengeAllDuringEvent
Challenge
Challenges every challengeable request during an event
ChallengeDDoSRequests
Challenge
Challenges suspected requests at your chosen sensitivity (only evaluated if the rule above is set to Count)
DDoSRequests
Block
Blocks suspected requests at or above your block sensitivity
Practical notes:
  • Only  GET  requests can be challenged. APIs and non-browser clients can't solve a Challenge, so exempt API paths using the exempt URI regex. The regex matches the host plus normalised path, without scheme or query string, so  ^/api/  won't work the way you expect. Test it.
  • If your app is sensitive to sudden spikes (a news site, ticket sales), keep  ChallengeAllDuringEvent  on. For a steadier app, set it to Count and tune  ChallengeDDoSRequests  instead.
  • It carries an additional charge. Check the AWS WAF pricing page for current rates.


CAPTCHA, Challenge and custom responses

Not everything suspicious should be blocked.
  • Challenge is a silent background check that confirms the client is a real browser. The user sees nothing (or a brief interstitial). Use it for suspicious-but-maybe-human traffic.
  • CAPTCHA shows a puzzle. Use it where friction is acceptable and the cost of a bot is high, such as login, sign-up, OTP and checkout.
  • Both work only for browsers on HTTPS. Never put them on API endpoints that mobile apps or servers call. Those clients can't solve them and will just break.
  • A solved CAPTCHA or Challenge gives the client a token valid for the immunity time (300 seconds by default), so users aren't challenged on every request.
  • CAPTCHA is billed per attempt ($0.40 per 1,000 attempts at the time of writing).
Custom responses make blocks useful rather than mysterious. Return  403  with a branded page and a reference ID your support team can search in the logs, and  429  with  Retry-After  for rate limits. A support ticket that says "I got error ref A7F2" is worth a lot more than "the site is broken".

Logging (and getting logs into your SIEM)

Every web ACL can send full logs to one destination, and the destination name must start with  aws-waf-logs- .
Destination
Best for
Trade-off
Amazon S3
Long-term retention and cost
Cheapest. Delivered in batches (around every 5 minutes). Query with Amazon Athena
CloudWatch Logs
Live troubleshooting, Logs Insights queries, alarms
Fastest to query, usually the most expensive at volume
Amazon Data Firehose
Sending logs to a SIEM (Splunk, OpenSearch, Wazuh and others) and transforming them
Extra Firehose cost. The stream must use Direct PUT as its source. Keep an S3 backup of everything
How I set it up for enterprises:
  • Production with a SOC: WAF → Firehose → SIEM, with Firehose backing everything up to S3. The SOC gets real-time alerts and you keep a cheap archive.
  • Cost-sensitive accounts: WAF → S3, plus Athena for investigations. Use S3 lifecycle rules to move old logs to cheaper storage classes.
  • Log filtering: keep only  BLOCK ,  COUNT ,  CAPTCHA  and  CHALLENGE  decisions if allowed traffic is too expensive to log. But keep everything during the 14-day tuning phase. You need the allowed traffic to spot what your Count rules would have broken.
  • Redact sensitive fields: the  Authorization  header and session cookies have no place in a log bucket.


Monitoring

Logs tell you what happened. Monitoring tells you while it's happening. Here's what I set up on every web ACL.

CloudWatch metrics and dashboards

AWS WAF publishes CloudWatch metrics for every rule that has metrics enabled:  AllowedRequests ,  BlockedRequests ,  CountedRequests ,  CaptchaRequests  and  ChallengeRequests , per rule and per web ACL. They're free.
Build one CloudWatch dashboard per web ACL with:
  • Total vs blocked requests over time (your "is something happening?" graph)
  • Blocked requests by rule (which defence is doing the work)
  • Counted requests by rule (what would break if you enforced it)
  • Rate-limit and Bot Control actions

Alarms that matter

  • Spike in blocked requests: an anomaly detection alarm on  BlockedRequests  for the web ACL. A sudden jump usually means an attack or a bad deploy that's now tripping rules.
  • Spike on the login rate limit: credential stuffing usually shows up here first.
  • Drop to zero allowed requests: if allowed traffic falls off a cliff, the WAF may be blocking everyone. This is the alarm that saves you from a self-inflicted outage.
Send them to SNS, and from there to your on-call tool or Slack.

The console views you'll actually use

  • Sampled requests: a rolling sample of up to 3 hours of requests that matched each rule, with headers. It's the fastest way to see why something was blocked.
  • Traffic overview dashboards: top blocked IPs, countries, URIs, bots and rules, out of the box.
  • AI activity dashboard: which AI bots and agents are hitting you, how often, on which paths, and whether they're verified.

A query for day one

In CloudWatch Logs Insights (or the equivalent in Athena or your SIEM), this shows which Count-mode rules would fire if you switched them to Block, and on which URIs:
Plain text
fields @timestamp, httpRequest.clientIp, httpRequest.uri, nonTerminatingMatchingRules.0.ruleId as countRule
| filter ispresent(nonTerminatingMatchingRules.0.ruleId)
| stats count(*) as hits by countRule, httpRequest.uri
| sort hits desc
| limit 50

If one rule fires thousands of times on  /api/upload , you've found your exception before your customers did.

Feed the SOC

In the SIEM, I create detections for: repeated blocks from one IP across multiple rules (a scanner), SQLi or Log4j matches on authenticated paths (a more targeted attacker), and any hit on the admin protection rule from outside your allow-listed ranges.


Fine-tuning: 14 days in Count mode, then enforce

This is the part that separates a WAF that protects from a WAF that gets switched off.
Day 0: deploy everything in Count mode. Every custom rule and every managed rule group, overridden to Count (for a rule group, set the override action to Count). Logging on. Nothing is blocked, but every match is recorded and labelled.
Days 1–14: watch real traffic. Two weeks covers weekday and weekend patterns, month-end jobs and most integrations. Each day, look at:
  • Which rules match, and on which URIs
  • Whether the matches are attacks (scanners, injection strings, Tor) or your own users (uploads, editors, partner IPs)
  • Rate-limit counts: is your threshold too tight for real users?
Day 14: switch to enforce, rule by rule.
  • High-confidence rules first: IP reputation, Known bad inputs, HTTP methods, geo.
  • Then the CRS, SQLi and OS rules, with the label-based exceptions you found.
  • Then rate limits, using numbers from real traffic plus a margin.
  • Bot Control and Anti-DDoS last, starting with Challenge and CAPTCHA before Block.
After go-live:
  • Set CloudWatch alarms on spikes in blocked requests per rule.
  • Review new managed rule group versions before adopting them. The console shows available versions.
  • Re-run the Count exercise whenever the app ships a big new feature.

Fixing the two classic false positives (the right way)

In my experience, two CRS rules cause most of the "WAF broke our app" tickets.
1.  SizeRestrictions_BODY  blocks request bodies over 8 KB. That breaks file uploads, large forms and big JSON payloads.
2.  CrossSiteScripting_BODY  fires on legitimate HTML from rich-text editors, CMS pages and email templates.
The wrong fix is to turn the whole rule group to Count, or remove the rule. The right fix is a targeted exception using labels:
  1. In the CRS rule group, override just those two rules to Count. Each match still adds a label, for example  awswaf:managed:aws:core-rule-set:SizeRestrictions_Body .
  1. Add a custom rule after the CRS that blocks requests carrying the label unless they're on the paths that legitimately need it.
JSON
{
  "Name": "CRS-SizeRestriction-ExceptUploads",
  "Priority": 21,
  "Statement": {
    "AndStatement": {
      "Statements": [
        { "LabelMatchStatement": { "Scope": "LABEL", "Key": "awswaf:managed:aws:core-rule-set:SizeRestrictions_Body" } },
        {
          "NotStatement": {
            "Statement": {
              "ByteMatchStatement": {
                "FieldToMatch": { "UriPath": {} },
                "PositionalConstraint": "STARTS_WITH",
                "SearchString": "/api/upload",
                "TextTransformations": [{ "Priority": 0, "Type": "LOWERCASE" }]
              }
            }
          }
        }
      ]
    }
  },
  "Action": { "Block": {} },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "CRS-SizeRestriction-ExceptUploads"
  }
}

Do the same for  CrossSiteScripting_Body , scoped to your CMS or editor endpoints. And keep the exception narrow: one path, ideally one method, and if possible only for authenticated users (for example, requests that carry your session cookie).
Label-based exceptions like this are the single most useful WAF skill. The managed rule still runs everywhere else, and you've documented exactly where and why you made an exception.



What it costs

AWS WAF pricing (standard, at the time of writing; Regional prices may vary):
Item
Price
Web ACL
$5.00 per month
Rule or rule group in the web ACL
$1.00 per month each
Requests
$0.60 per million
WCUs above 1,500
$0.20 per million requests per extra 500 WCUs
Body inspection above default
$0.30 per million requests per extra 16 KB
Bot Control Common
$10/month + $1 per million requests (first 10M/month free)
Bot Control Targeted
$10/month + $10 per million requests (first 1M/month free)
CAPTCHA
$0.40 per 1,000 attempts
Logging
500 MB of CloudWatch/Vended Logs per 1M requests included, then Vended Logs rates
AI activity dashboard and AI traffic monetisation
No additional charge
A worked example. One web ACL with 6 custom rules and 10 managed rule groups, serving 50 million requests a month, with Bot Control Common scoped down to dynamic paths that see 30 million of those requests:
  • Web ACL: $5
  • 16 rules and rule groups: $16
  • Requests: 50 × $0.60 = $30
  • WCUs: around 1,950, so one extra 500-WCU block: 50 × $0.20 = $10
  • Bot Control Common: $10 + (30 − 10) × $1 = $30
  • Total: about $91 per month, plus logging, CAPTCHA attempts and the Anti-DDoS rule group if you use them.
If that's more than you can justify today, start with the must-have rules from Part 1 (about $12 a month plus requests) and add the paid groups once the logs show you need them.
Compare that with the cost of one credential-stuffing weekend.


The checklist

If you only take one thing from this post, take this:
  • [ ] Web ACL on CloudFront (global) where possible, plus a regional one on the ALB
  • [ ] IP allow-list and block-list
  • [ ] Geo allow-list for your business countries
  • [ ] Only the HTTP methods your app uses
  • [ ] Global and login rate limits with a 429 response
  • [ ] File-type allow-list on upload endpoints
  • [ ] IP reputation and anonymous IP lists
  • [ ] Core rule set, Known bad inputs, SQLi, and the OS and app groups for your stack
  • [ ] Label-based exceptions for  SizeRestrictions_BODY  and  CrossSiteScripting_BODY , never whole-group Count
  • [ ] Bot Control, scoped down, with Targeted on high-value paths
  • [ ] Anti-DDoS rule group with API paths exempted
  • [ ] Logging to S3 (cheap) or Firehose → SIEM, with sensitive fields redacted
  • [ ] 14 days in Count, then enforce rule by rule
  • [ ] CloudWatch alarms on blocked-request spikes
A WAF that's tuned to your business blocks more attacks and breaks fewer customers than one that just has every box ticked. Take the two weeks. It's worth it.
Have a WAF tuning war story, or a rule you always add that's not on this list? Tell me in the comments.


References

 #AWS   #AWSWAF   #CloudSecurity   #WebApplicationFirewall   #AppSec   #DevSecOps 
# aws-waf# cloud-security# security-best-practices# application-security# amazon-cloudwatch
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article