AWS Builder Center

Fool price-scraping bots with fake data using AWS WAF Bot Control

Fool price-scraping bots with fake data using AWS WAF Bot Control

Bots scraping your product catalog don't announce themselves. They grab your pricing, feed it to a competitor's algorithm, and you're left wondering why you keep getting undercut on every deal. Blocking them outright sounds clean, but it tips them off. They rotate IPs, switch user agents, add delays. The arms race escalates.
What if you let them in, but served garbage?
In this post, I show how to build a response manipulation layer using  AWS WAF  Bot Control that detects scraper traffic and silently routes it to a decoy backend serving fake pricing data. The bot thinks it succeeded. Your real customers see the real numbers. And you bought yourself time before the attacker realizes something's off.

The idea

I work with retail and financial services customers in Southeast Asia. Price scraping is a constant headache. A competitor's bot hits your API every few minutes, pulls your catalog, and adjusts their pricing in near-real-time. The conventional approach is to block at the WAF level. That works until it doesn't.
The problem with blocking: a sophisticated scraper operator notices the 403 within minutes. They'll cycle through residential proxies, emulate browser fingerprints, and distribute requests across hundreds of IPs. You're now maintaining an increasingly complex ruleset that requires weekly tuning.
The deception approach flips the dynamic. Instead of blocking, you let the request through but serve modified data. The scraper sees a 200 response with valid-looking HTML. Their pipeline ingests it without triggering any error handling. They only discover the deception when their pricing strategy fails in market, which could take weeks.

Architecture overview


Figure 1: Bot traffic gets labeled by AWS WAF Bot Control, receives a custom header, and ALB routes it to the decoy Lambda.
The flow:
  1. All traffic enters through  Amazon CloudFront  with AWS WAF attached.
  1. AWS WAF Bot Control (Common level) inspects requests. If it matches the  CategoryHttpLibrary  rule, it labels the request with  awswaf:managed:aws:bot-control:bot:category:http_library .
  1. A custom rule matches on that label and adds header  x-amzn-waf-header-bot: bot  via the Count action with custom request handling.
  1.  Application Load Balancer  (ALB) inspects headers: requests carrying  x-amzn-waf-header-bot  route to the decoy target group. Everything else goes to the real application.
  1. The decoy Lambda returns believable but wrong prices.
AWS services used:
Service
Role
Amazon CloudFront
Entry point, caching, WAF integration
AWS WAF + Bot Control
Bot detection, labeling, custom header injection
Application Load Balancer
Header-based routing to different backends
AWS Lambda (×2)
Real application and decoy application

Prerequisites

  • An AWS account with permissions for CloudFront, WAF, ALB, and Lambda
  • A VPC with at least two public subnets across different Availability Zones
  • Python 3.12 runtime (Lambda)
  • AWS CLI or Console access in  ap-southeast-1  (or your preferred region)

How I built it

Step 1: Two Lambda functions with different prices

The real application returns actual pricing:
def lambda_handler(event, context):
products = [
{"id": 1, "name": "Table", "price": 199.99},
{"id": 2, "name": "Chair", "price": 89.99}
]

HTML_TEMPLATE = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Product List</title>
<style>
body {{ font-family: Arial, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; }}
h1 {{ color: #333; }}
ul {{ list-style-type: none; padding: 0; }}
li {{ margin-bottom: 10px; }}
</style>
</head>
<body>
<h1>Product List</h1>
<ul id="product-list">
{product_list}
</ul>
</body>
</html>
"""
product_list = "\n".join(
[f"<li>{p['name']} - ${p['price']}</li>" for p in products]
)
html = HTML_TEMPLATE.format(product_list=product_list)
return {
'statusCode': 200,
'headers': {'Content-Type': 'text/html'},
'body': html
}
The decoy function is identical except for the numbers. I inflated prices by 3x on the table and 2x on the chair. Enough to look plausible (it's not $0.01 or $99,999) but enough to wreck any pricing algorithm relying on this data.
products = [
{"id": 1, "name": "Table", "price": 599.99},
{"id": 2, "name": "Chair", "price": 189.99}
]
Create both functions in the Lambda console. Name them  lambda-original-app  and  lambda-fake-app . Python 3.12, default settings for everything else.

Step 2: ALB with header-based routing

Create two Lambda target groups:
  •  lambda-original-tg  pointing to  lambda-original-app 
  •  lambda-fake-tg  pointing to  lambda-fake-app 
Enable health checks with the default  /  path on both.
Create an internet-facing ALB named  product-price-alb :
  • Scheme: Internet-facing, IPv4
  • Map to at least 2 AZs with public subnets
  • Security group allowing inbound HTTP port 80
  • Default listener rule (HTTP:80) forwards to  lambda-original-tg 
Then add a second listener rule with priority 1:
  • Condition: HTTP Header  x-amzn-waf-header-bot  equals  bot 
  • Action: Forward to  lambda-fake-tg 
Test it with curl:
# Normal request → real prices
curl product-price-alb-261837xxx.ap-southeast-1.elb.amazonaws.com

# Output: Table - $199.99, Chair - $89.99

# Simulate bot header → fake prices
curl --header "x-amzn-waf-header-bot:bot" product-price-alb-261837xxx.ap-southeast-1.elb.amazonaws.com

# Output: Table - $599.99, Chair - $189.99
This confirms the routing logic works before we wire up WAF.

Step 3: CloudFront distribution with WAF

Create a new CloudFront distribution:
  • Origin domain: your ALB DNS name
  • Origin protocol: HTTP only
  • Under "Web Application Firewall (WAF)": select Enable security protections
  • Leave everything else default
CloudFront auto-creates a WebACL with name prefix  CreatedByCloudFront- . You'll find it in the WAF console under the Global (CloudFront) scope.

Step 4: Bot Control rules and custom header injection

This is where the real plumbing happens.
Add Bot Control managed rule group:
  1. Open the auto-created WebACL in the AWS WAF console.
  1. Go to Rules → Add rules → Add managed rule groups.
  1. Enable Bot Control. Set inspection level to Common. Scope: inspect all web requests.
  1. On the  CategoryHttpLibrary  and  CategoryScrapingFramework  rules, override the action to Count.
Why Count instead of Block? The Count action is non-terminating. It lets the request continue through subsequent rules while adding a label. That label ( awswaf:managed:aws:bot-control:bot:category:http_library ) becomes available for matching in your custom rules downstream.
Create custom header-injection rules:
Add a custom rule named  header-bot-rule :
  • Type: Regular rule
  • Statement: Has a label → Match scope: Label → Match key:  awswaf:managed:aws:bot-control:bot:category:http_library 
  • Action: Count
  • Custom request handling: add header with key  header-bot  and value  bot 
AWS WAF automatically prefixes your header name with  x-amzn-waf- , so the downstream ALB sees  x-amzn-waf-header-bot: bot . This prefix prevents clients from spoofing WAF-injected headers. (You should also add a rule early in your WebACL that blocks any incoming request already carrying an  x-amzn-waf-  header. The  documentation  shows a string match rule for this.)
Create a second rule  header-bot-rule-scraping  with the same configuration but matching on label  awswaf:managed:aws:bot-control:bot:category:scraping_framework .
Make sure both custom rules have lower priority numbers (higher priority) than the Bot Control managed rule group, since labels must be generated before they can be matched.
Wait. Actually, that's backwards. I got this wrong the first time. The Bot Control managed rule group needs to run first (higher priority, meaning lower priority number) to generate the labels. Then your custom rules run after to match those labels. If you put the custom rules first, there are no labels to match yet. Set priorities so Bot Control is priority 1, then your custom rules at 2 and 3.

Results

Open a browser and navigate to your CloudFront distribution URL ( https://dXXXXXXXXXXXw.cloudfront.net/ ). You'll see the real prices: Table $199.99, Chair $89.99.
Now from a terminal:
curl https://dXXXXXXXXXXXw.cloudfront.net/
The response shows inflated prices: Table $599.99, Chair $189.99. Because curl is an HTTP library, Bot Control labels it, the custom rule injects the header, and ALB routes to the decoy.
In the WAF console under Traffic overview, you can see curl requests flagged as bot traffic matching the  CategoryHttpLibrary  signal.
The bot operator sees a clean 200 response with valid HTML. Their scraping pipeline ingests it without error. No 403, no CAPTCHA, no indication anything unusual happened.

What I'd do differently

The decoy application in this tutorial is laughably simple. Two hardcoded products. In production, you'd want the decoy to mirror the real app's structure exactly but apply a transformation layer on prices (random multiplier between 1.5x and 4x, perhaps seeded by product ID so the same product always returns the same fake price for consistency).
One thing that caught me off guard: CloudFront caching. If you don't set appropriate cache headers, CloudFront might cache the decoy response and serve it to real users on subsequent requests. I had to set  Cache-Control: no-cache  on the decoy Lambda responses and configure the CloudFront behavior to forward the  x-amzn-waf-header-bot  header to origin. Otherwise the cache key doesn't differentiate between bot and human requests.
You should also log requests hitting the decoy backend. If the decoy gets zero traffic for a week after deployment, it means either your bots already moved to a more sophisticated approach (browser automation instead of HTTP libraries) or your label matching isn't catching them. Either way, you need to know.
The Common level of Bot Control only catches unsophisticated bots: things using  requests ,  urllib ,  curl , or known scraping frameworks. For browser-automated bots using Playwright or Puppeteer, you'd need the Targeted protection level, which uses JavaScript challenges and behavioral analysis. That costs $10 per million requests instead of $1 per million (source:  AWS WAF Pricing API , queried 2026-08-17 for ap-southeast-1).

Cost

For ap-southeast-1, based on the AWS Pricing API (queried 2026-08-17):
Component
Unit cost
Source
WebACL
$5.00/month
AWS Pricing API,  awswaf ,  APS1-WebACLV2 
Rule (×4 rules: default + Bot Control + 2 custom)
$1.00/month each
AWS Pricing API,  awswaf ,  APS1-RuleV2 
Bot Control Common requests
$1.00 per 1M requests
AWS Pricing API,  awswaf ,  APS1-AMR-BotControl-Request 
Monthly estimate at 10M requests: $5 (WebACL) + $4 (rules) + $10 (Bot Control) = $19/month. The decoy Lambda itself costs virtually nothing since it's only invoked for bot traffic.

Other considerations

This approach uses  CategoryHttpLibrary  and  CategoryScrapingFramework  for detection. Those catch the low-hanging fruit. For more advanced protection, consider combining with:
  • Cloud IP reputation rules: flags requests from known cloud provider IP ranges
  • Rate-based rules: catches volumetric scraping patterns from a single source
  • Targeted Bot Control: uses ML and JavaScript challenges for browser-automated bots
As an alternative architecture, you could place the WAF and routing at the API Gateway or ALB layer directly instead of CloudFront. The custom header injection feature works identically on all three resource types that AWS WAF supports.
Keep the decoy data plausible. If your real product is $199 and the decoy returns $99,999, even a basic sanity check in the scraper's pipeline will flag it. Aim for prices that are wrong enough to damage their pricing strategy but close enough to pass automated validation.

Clean up resources

Delete these resources to avoid ongoing charges:
  1. CloudFront distribution (disable first, then delete)
  1. WAF WebACL (disassociate from CloudFront first)
  1. ALB and both target groups
  1. Both Lambda functions

Try it yourself

The full implementation takes about 30 minutes through the console. The core concept, using WAF labels to inject custom headers that drive routing decisions, applies to any scenario where you want different backends for different traffic classifications without revealing the differentiation to the client.
Start with the  AWS WAF Bot Control documentation  and the  custom request header insertion guide . The free  AWS WAF module on AWS Skill Builder  covers the fundamentals if you're starting fresh.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article