AWS Builder Center

Builder Center

WAF Log Circuit Breaker: Cost-Bounded Logging for High-Volume Traffic

WAF Log Circuit Breaker: Cost-Bounded Logging for High-Volume Traffic

AWS WAF log filtering is great for saving costs, but what happens if you still want to log your HTTP traffic, but avoid cost spikes for highly volumetric traffic? Fear not, solutions ahead!

Tzoori Tamam

You're under attack. Or maybe you're not — maybe it's just a traffic spike. Either way, you need logs to understand what's happening.
But here's the problem: logging everything at scale can come at a cost. A million requests per minute flowing into CloudWatch Logs, S3, or Firehose can add up to thousands of dollars per month. So you turn off logging. And now you're blind.
What if you could log just enough to understand a pattern — and automatically stop once you've captured what you need?
This post introduces a pattern that gives you cost-bounded WAF logging using nothing but native WAF features. No Lambda. No post-processing. No surprises on your bill.

The problem

AWS WAF can log every request it evaluates. That's powerful — until your traffic volume makes it expensive. For high-traffic applications — APIs handling tens of thousands of requests per second — full logging can be a financial pain. But selective logging (WAF log filtering) has a gap: you can filter what to log, but you can't control how much of it gets logged.
Until now.

The building blocks

WAF gives us three features that, combined, solve this:
  1. Count rules with labels — classify traffic without blocking it. Any rule in Count mode can attach a label to matching requests.
  2. Rate-based rules with labels — track request volume for traffic matching a condition. When the volume exceeds a threshold, the rule fires and adds its own label. Crucially: this works in Count mode too — no blocking required.
  3. Log filtering — include or exclude requests from logs based on label presence. This is the gate that decides what actually gets written.
Individually, these are standard WAF features. Together, they form a log sampling circuit breaker.

The pattern

Step 1: Label the traffic you care about

Create WAF rules (Count mode) that label interesting traffic patterns:
  • Requests from a specific geo that you're investigating
  • Requests matching a suspicious header pattern
  • Requests hitting a sensitive endpoint
  • Bot traffic you want to observe (if you're using Bot Control, its labels are available for matching)
Each rule adds a label like logging:keep-geo, logging:keep-probes, or logging:keep-bots.

Step 2: Rate-limit the logging signal

Create a rate-based rule that:
  • Scopes to: requests that already have your logging labels (using a scope-down statement matching on label)
  • Aggregation: Count All (no per-IP or per-header aggregation — you're measuring total volume of the pattern)
  • Threshold: your budget tolerance (e.g., 10,000 requests per 60 seconds)
  • Action: Count (not block!)
  • Label: logging:override-drop (or log:suppress, or whatever makes sense to you)
This rule does nothing to the traffic. It just watches the volume of your "interesting" requests and fires a label when they exceed your threshold.

Step 3: Wire the log filter

Configure your WAF log filter with:
  • Default behavior: Drop (don't log anything by default)
  • Filter rule: Keep requests that have logging labels AND do NOT have the logging:override-drop label
1
2
3
Log filter condition:
DROP when: hasLabel("logging:override-drop")
KEEP when: hasLabel("logging:keep")

The cost math

For context: 10 million requests per month in CloudWatch Logs costs roughly $5. But at 10 billion, that's thousands of dollars — and that's just ingestion.
Let's say you're investigating bot traffic that normally runs at 50 requests/second but spikes to 10,000 requests/second during attacks.
Without sampling:
  • 10,000 req/s × 3,600s × cost = massive CloudWatch bill for data you've already characterized
With this pattern (threshold: 10,000 per 60 seconds):
  • You log the first ~10,000 requests before the rate threshold kicks in
  • Once volume exceeds the threshold, the rate-based rule fires and logging stops
  • Your cost is bounded to the initial burst, regardless of how long the attack lasts
You've turned an unpredictable, traffic-proportional cost into a fixed ceiling.

Configuration example

Let's say you run a SaaS platform with tiered customers. Your diamond-tier customers get priority support, and when they report issues, you want WAF logs readily available for troubleshooting. But diamond customers also generate the most traffic — they're your power users. During peak hours, logging all their requests would be expensive and redundant.
Here's how to set it up: log diamond-tier traffic by default, but automatically suppress logging when their volume spikes beyond what's useful for debugging.

Rule 1: Label diamond-tier traffic (Count)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"Name": "label-high-interest-traffic-patterns",
"Priority": 0,
"Statement": {
"ByteMatchStatement": {
"SearchString": "diamond",
"FieldToMatch": {
"SingleQueryArgument": {
"Name": "customer-type"
}
},
"TextTransformations": [
{ "Priority": 0, "Type": "NONE" }
],
"PositionalConstraint": "EXACTLY"
}
},
"Action": { "Count": {} },
"RuleLabels": [
{ "Name": "logging:keep" }
],
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "label-high-interest-traffic-patterns"
}
}

Rule 2: Rate-based volume detection (Count)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
"Name": "throttle-volumetric-logging",
"Priority": 1,
"Statement": {
"RateBasedStatement": {
"Limit": 10000,
"EvaluationWindowSec": 60,
"AggregateKeyType": "CONSTANT",
"ScopeDownStatement": {
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "logging:keep"
}
}
}
},
"Action": { "Count": {} },
"RuleLabels": [
{ "Name": "logging:override-drop" }
],
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "throttle-volumetric-logging"
}
}

Log filter configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"DefaultBehavior": "DROP",
"Filters": [
{
"Behavior": "DROP",
"Requirement": "MEETS_ANY",
"Conditions": [
{
"LabelNameCondition": {
"LabelName": "logging:override-drop"
}
}
]
},
{
"Behavior": "KEEP",
"Requirement": "MEETS_ALL",
"Conditions": [
{
"LabelNameCondition": {
"LabelName": "logging:keep"
}
}
]
}
]
}
The first filter says "drop anything with the suppression label." The second says "keep requests labeled logging:keep." Since the first matching filter wins, the DROP takes priority when both labels are present.

Important detail: how rate-based rules apply labels

When a rate-based rule fires, it adds the label to all subsequent requests matching the scope-down statement — not just the ones above the threshold. This means once the rate is exceeded, every request matching the pattern gets the logging:override-drop label until the rate drops back below threshold.
This is exactly what we want. The log filter sees:
  • Below threshold: requests have logging:keep but NOT logging:override-drop → logged
  • Above threshold: requests have both logging:keep AND logging:override-drop → dropped
When the rate drops, the rule stops firing, labels stop being applied, and logging resumes.
One thing to keep in mind: for traffic patterns that are consistently volumetric (they never drop below your threshold), logging will only capture the initial burst and won't resume until the volume actually subsides. For most investigation scenarios this is fine — you rarely need more than a few thousand requests to characterize repetitive traffic.

When to use this

This pattern works best when:
  • You have high-traffic applications where full logging is cost-prohibitive
  • You need visibility into specific traffic patterns (bots, anomalies, specific geos)
  • Attack traffic is repetitive (once you've seen 1,000 requests, you've seen them all)
  • You want cost predictability without sacrificing observability
It's less useful when:
  • Every request is unique and needs individual forensics (financial transactions)
  • You need complete audit trails for compliance
  • Traffic volume is already low enough that full logging is affordable

Beyond single patterns

You can layer multiple independent sampling circuits:
  • logging:keep + logging:override-drop:bots (threshold: 5,000)
  • logging:keep-geo + logging:override-drop:geo (threshold: 2,000)
  • logging:keep-auth + logging:override-drop:auth (threshold: 1,000)
Each pattern gets its own volume threshold and suppression label. High-volume bot traffic won't suppress your auth failure logs, and vice versa.

Summary

WAF log filtering lets you choose what to log. Rate-based rules let you control how much. Together, they give you something that didn't exist before: self-regulating, cost-bounded observability at the edge.
You see what you need to see. You stop paying when you've seen enough. And when the storm passes, the window opens again.
No Lambda functions. No post-processing pipelines. No bill shock. Just WAF rules talking to each other through labels.
Tzoori Tamam is a Principal Solutions Architect at AWS, specializing in Edge Services.
# aws-waf# waf# aws-edge-services# cost# cost-optimization
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article