Fool price-scraping bots with fake data using AWS WAF Bot Control
Fool price-scraping bots with fake data using AWS WAF Bot Control
Bots scraping your product catalog don't announce themselves. They grab your pricing, feed it to a competitor's algorithm, and you're left wondering why you keep getting undercut on every deal. Blocking them outright sounds clean, but it tips them off. They rotate IPs, switch user agents, add delays. The arms race escalates.
What if you let them in, but served garbage?
In this post, I show how to build a response manipulation layer using AWS WAF Bot Control that detects scraper traffic and silently routes it to a decoy backend serving fake pricing data. The bot thinks it succeeded. Your real customers see the real numbers. And you bought yourself time before the attacker realizes something's off.
The idea
I work with retail and financial services customers in Southeast Asia. Price scraping is a constant headache. A competitor's bot hits your API every few minutes, pulls your catalog, and adjusts their pricing in near-real-time. The conventional approach is to block at the WAF level. That works until it doesn't.
The problem with blocking: a sophisticated scraper operator notices the 403 within minutes. They'll cycle through residential proxies, emulate browser fingerprints, and distribute requests across hundreds of IPs. You're now maintaining an increasingly complex ruleset that requires weekly tuning.
The deception approach flips the dynamic. Instead of blocking, you let the request through but serve modified data. The scraper sees a 200 response with valid-looking HTML. Their pipeline ingests it without triggering any error handling. They only discover the deception when their pricing strategy fails in market, which could take weeks.
Architecture overview

Figure 1: Bot traffic gets labeled by AWS WAF Bot Control, receives a custom header, and ALB routes it to the decoy Lambda.
The flow:
- All traffic enters through Amazon CloudFront with AWS WAF attached.
- AWS WAF Bot Control (Common level) inspects requests. If it matches the
CategoryHttpLibraryrule, it labels the request withawswaf:managed:aws:bot-control:bot:category:http_library.
- A custom rule matches on that label and adds header
x-amzn-waf-header-bot: botvia the Count action with custom request handling.
- Application Load Balancer (ALB) inspects headers: requests carrying
x-amzn-waf-header-botroute to the decoy target group. Everything else goes to the real application.
- The decoy Lambda returns believable but wrong prices.
AWS services used:
Service | Role |
|---|---|
Amazon CloudFront | Entry point, caching, WAF integration |
AWS WAF + Bot Control | Bot detection, labeling, custom header injection |
Application Load Balancer | Header-based routing to different backends |
AWS Lambda (×2) | Real application and decoy application |
Prerequisites
- An AWS account with permissions for CloudFront, WAF, ALB, and Lambda
- A VPC with at least two public subnets across different Availability Zones
- Python 3.12 runtime (Lambda)
- AWS CLI or Console access in
ap-southeast-1(or your preferred region)
How I built it
Step 1: Two Lambda functions with different prices
The real application returns actual pricing:
def lambda_handler(event, context): products = [ {"id": 1, "name": "Table", "price": 199.99}, {"id": 2, "name": "Chair", "price": 89.99} ]
HTML_TEMPLATE = """ <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Product List</title> <style> body {{ font-family: Arial, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; }} h1 {{ color: #333; }} ul {{ list-style-type: none; padding: 0; }} li {{ margin-bottom: 10px; }} </style> </head> <body> <h1>Product List</h1> <ul id="product-list"> {product_list} </ul> </body> </html> """ product_list = "\n".join( [f"<li>{p['name']} - ${p['price']}</li>" for p in products] ) html = HTML_TEMPLATE.format(product_list=product_list) return { 'statusCode': 200, 'headers': {'Content-Type': 'text/html'}, 'body': html }The decoy function is identical except for the numbers. I inflated prices by 3x on the table and 2x on the chair. Enough to look plausible (it's not $0.01 or $99,999) but enough to wreck any pricing algorithm relying on this data.
products = [ {"id": 1, "name": "Table", "price": 599.99}, {"id": 2, "name": "Chair", "price": 189.99} ]Create both functions in the Lambda console. Name them
lambda-original-app and lambda-fake-app . Python 3.12, default settings for everything else.Step 2: ALB with header-based routing
Create two Lambda target groups:
-
lambda-original-tgpointing tolambda-original-app
-
lambda-fake-tgpointing tolambda-fake-app
Enable health checks with the default
/ path on both.Create an internet-facing ALB named
product-price-alb :- Scheme: Internet-facing, IPv4
- Map to at least 2 AZs with public subnets
- Security group allowing inbound HTTP port 80
- Default listener rule (HTTP:80) forwards to
lambda-original-tg
Then add a second listener rule with priority 1:
- Condition: HTTP Header
x-amzn-waf-header-botequalsbot
- Action: Forward to
lambda-fake-tg
Test it with curl:
# Normal request → real pricescurl product-price-alb-261837xxx.ap-southeast-1.elb.amazonaws.com
# Output: Table - $199.99, Chair - $89.99
# Simulate bot header → fake pricescurl --header "x-amzn-waf-header-bot:bot" product-price-alb-261837xxx.ap-southeast-1.elb.amazonaws.com
# Output: Table - $599.99, Chair - $189.99This confirms the routing logic works before we wire up WAF.
Step 3: CloudFront distribution with WAF
Create a new CloudFront distribution:
- Origin domain: your ALB DNS name
- Origin protocol: HTTP only
- Under "Web Application Firewall (WAF)": select Enable security protections
- Leave everything else default
CloudFront auto-creates a WebACL with name prefix
CreatedByCloudFront- . You'll find it in the WAF console under the Global (CloudFront) scope.Step 4: Bot Control rules and custom header injection
This is where the real plumbing happens.
Add Bot Control managed rule group:
- Open the auto-created WebACL in the AWS WAF console.
- Go to Rules → Add rules → Add managed rule groups.
- Enable Bot Control. Set inspection level to Common. Scope: inspect all web requests.
- On the
CategoryHttpLibraryandCategoryScrapingFrameworkrules, override the action to Count.
Why Count instead of Block? The Count action is non-terminating. It lets the request continue through subsequent rules while adding a label. That label (
awswaf:managed:aws:bot-control:bot:category:http_library ) becomes available for matching in your custom rules downstream.Create custom header-injection rules:
Add a custom rule named
header-bot-rule :- Type: Regular rule
- Statement: Has a label → Match scope: Label → Match key:
awswaf:managed:aws:bot-control:bot:category:http_library
- Action: Count
- Custom request handling: add header with key
header-botand valuebot
AWS WAF automatically prefixes your header name with
x-amzn-waf- , so the downstream ALB sees x-amzn-waf-header-bot: bot . This prefix prevents clients from spoofing WAF-injected headers. (You should also add a rule early in your WebACL that blocks any incoming request already carrying an x-amzn-waf- header. The documentation shows a string match rule for this.)Create a second rule
header-bot-rule-scraping with the same configuration but matching on label awswaf:managed:aws:bot-control:bot:category:scraping_framework .Make sure both custom rules have lower priority numbers (higher priority) than the Bot Control managed rule group, since labels must be generated before they can be matched.
Wait. Actually, that's backwards. I got this wrong the first time. The Bot Control managed rule group needs to run first (higher priority, meaning lower priority number) to generate the labels. Then your custom rules run after to match those labels. If you put the custom rules first, there are no labels to match yet. Set priorities so Bot Control is priority 1, then your custom rules at 2 and 3.
Results
Open a browser and navigate to your CloudFront distribution URL (
https://dXXXXXXXXXXXw.cloudfront.net/ ). You'll see the real prices: Table $199.99, Chair $89.99.Now from a terminal:
curl https://dXXXXXXXXXXXw.cloudfront.net/The response shows inflated prices: Table $599.99, Chair $189.99. Because curl is an HTTP library, Bot Control labels it, the custom rule injects the header, and ALB routes to the decoy.
In the WAF console under Traffic overview, you can see curl requests flagged as bot traffic matching the
CategoryHttpLibrary signal.The bot operator sees a clean 200 response with valid HTML. Their scraping pipeline ingests it without error. No 403, no CAPTCHA, no indication anything unusual happened.
What I'd do differently
The decoy application in this tutorial is laughably simple. Two hardcoded products. In production, you'd want the decoy to mirror the real app's structure exactly but apply a transformation layer on prices (random multiplier between 1.5x and 4x, perhaps seeded by product ID so the same product always returns the same fake price for consistency).
One thing that caught me off guard: CloudFront caching. If you don't set appropriate cache headers, CloudFront might cache the decoy response and serve it to real users on subsequent requests. I had to set
Cache-Control: no-cache on the decoy Lambda responses and configure the CloudFront behavior to forward the x-amzn-waf-header-bot header to origin. Otherwise the cache key doesn't differentiate between bot and human requests.You should also log requests hitting the decoy backend. If the decoy gets zero traffic for a week after deployment, it means either your bots already moved to a more sophisticated approach (browser automation instead of HTTP libraries) or your label matching isn't catching them. Either way, you need to know.
The Common level of Bot Control only catches unsophisticated bots: things using
requests , urllib , curl , or known scraping frameworks. For browser-automated bots using Playwright or Puppeteer, you'd need the Targeted protection level, which uses JavaScript challenges and behavioral analysis. That costs $10 per million requests instead of $1 per million (source: AWS WAF Pricing API , queried 2026-08-17 for ap-southeast-1).Cost
For ap-southeast-1, based on the AWS Pricing API (queried 2026-08-17):
Component | Unit cost | Source |
|---|---|---|
WebACL | $5.00/month | AWS Pricing API, awswaf , APS1-WebACLV2 |
Rule (×4 rules: default + Bot Control + 2 custom) | $1.00/month each | AWS Pricing API, awswaf , APS1-RuleV2 |
Bot Control Common requests | $1.00 per 1M requests | AWS Pricing API, awswaf , APS1-AMR-BotControl-Request |
Monthly estimate at 10M requests: $5 (WebACL) + $4 (rules) + $10 (Bot Control) = $19/month. The decoy Lambda itself costs virtually nothing since it's only invoked for bot traffic.
Other considerations
This approach uses
CategoryHttpLibrary and CategoryScrapingFramework for detection. Those catch the low-hanging fruit. For more advanced protection, consider combining with:- Token absent detection: blocks requests missing a valid AWS WAF token (requires the AWS WAF JavaScript SDK integration )
- Cloud IP reputation rules: flags requests from known cloud provider IP ranges
- Rate-based rules: catches volumetric scraping patterns from a single source
- Targeted Bot Control: uses ML and JavaScript challenges for browser-automated bots
As an alternative architecture, you could place the WAF and routing at the API Gateway or ALB layer directly instead of CloudFront. The custom header injection feature works identically on all three resource types that AWS WAF supports.
Keep the decoy data plausible. If your real product is $199 and the decoy returns $99,999, even a basic sanity check in the scraper's pipeline will flag it. Aim for prices that are wrong enough to damage their pricing strategy but close enough to pass automated validation.
Clean up resources
Delete these resources to avoid ongoing charges:
- CloudFront distribution (disable first, then delete)
- WAF WebACL (disassociate from CloudFront first)
- ALB and both target groups
- Both Lambda functions
Try it yourself
The full implementation takes about 30 minutes through the console. The core concept, using WAF labels to inject custom headers that drive routing decisions, applies to any scenario where you want different backends for different traffic classifications without revealing the differentiation to the client.
Start with the AWS WAF Bot Control documentation and the custom request header insertion guide . The free AWS WAF module on AWS Skill Builder covers the fundamentals if you're starting fresh.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article