I Was Polling My Database Every 5 Seconds. SQS + Lambda Made It Real-Time and Cut My Bill is $50
My EC2 cron job queried RDS every 5 seconds to check for new orders. It was expensive, slow, and still missed updates. I replaced it with SQS + Lambda; now it's instant, scales to zero, and costs $0.20.
The Dumb Thing I Did For 6 Months
My app had a simple flow: User places order -> Admin needs to get email.
My app had a simple flow: User places order -> Admin needs to get email.
My solution? A cron on EC2:
Bash
*/5 * * * * python check_new_orders.py # query RDS every 5 seconds
Result: 17,280 queries per day. 99.9% returned "no new orders." RDS CPU 40% just for polling. And email was still 5 seconds late.
*/5 * * * * python check_new_orders.py # query RDS every 5 seconds
Result: 17,280 queries per day. 99.9% returned "no new orders." RDS CPU 40% just for polling. And email was still 5 seconds late.
The Fix: Let AWS Tell Me When Something Happens
New flow: Order placed -> API sends message to SQS -> SQS triggers Lambda -> Lambda sends email
New flow: Order placed -> API sends message to SQS -> SQS triggers Lambda -> Lambda sends email
No polling. No wasted queries. Real-time.
Step 1: Create SQS Queue (30 seconds)
SQS > Create Queue > Standard > Name: new-orders-queue
Enable: Dead-Letter Queue create new-orders-dlq. This is critical. If your Lambda fails 3 times, the message goes to DLQ instead of looping forever and billing you.
SQS > Create Queue > Standard > Name: new-orders-queue
Enable: Dead-Letter Queue create new-orders-dlq. This is critical. If your Lambda fails 3 times, the message goes to DLQ instead of looping forever and billing you.
Step 2: Send Message Instead of Writing Status Flag
Instead of INSERT INTO orders... and waiting for cron to find it:
Instead of INSERT INTO orders... and waiting for cron to find it:
Python
import boto3
sqs = boto3.client('sqs')
queue_url = 'https://sqs.ap-south-1.amazonaws.com/123456789/new-orders-queue'
import boto3
sqs = boto3.client('sqs')
queue_url = 'https://sqs.ap-south-1.amazonaws.com/123456789/new-orders-queue'
sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({"order_id": 123, "email": "user@example.com"}),
MessageGroupId="orders" # for FIFO if you need an order
)
QueueUrl=queue_url,
MessageBody=json.dumps({"order_id": 123, "email": "user@example.com"}),
MessageGroupId="orders" # for FIFO if you need an order
)
4 lines hidden
Cost: $0.40 per 1M messages. My 500 orders/month = $0.0002.
Cost: $0.40 per 1M messages. My 500 orders/month = $0.0002.
Step 3: Lambda Triggered by SQS
Lambda > Create > Trigger > SQS > new-orders-queue
Batch size: 10, Window: 5 seconds
Lambda > Create > Trigger > SQS > new-orders-queue
Batch size: 10, Window: 5 seconds
Python
def lambda_handler(event, context):
for record in event['Records']:
order = json.loads(record['body'])
# Idempotency check! SQS can deliver twice.
if already_processed(order['order_id']):
continue
send_email(order['email'])
print(f"Processed {order['order_id']}")
def lambda_handler(event, context):
for record in event['Records']:
order = json.loads(record['body'])
# Idempotency check! SQS can deliver twice.
if already_processed(order['order_id']):
continue
send_email(order['email'])
print(f"Processed {order['order_id']}")
3 lines hidden
No server to manage. If 1000 orders come in 1 second, Lambda scales to 1000 parallel runs automatically.
No server to manage. If 1000 orders come in 1 second, Lambda scales to 1000 parallel runs automatically.
The Result
Before: EC2 t3.small 24/7 ($15) + RDS extra load ($35) = $50/month for polling
After: SQS ($0.0002) + Lambda 500 invocations ($0.02) = $0.22/month
And email goes from a 5-second delay to <400 ms. My RDS CPU dropped from 40% to 4%.
Before: EC2 t3.small 24/7 ($15) + RDS extra load ($35) = $50/month for polling
After: SQS ($0.0002) + Lambda 500 invocations ($0.02) = $0.22/month
And email goes from a 5-second delay to <400 ms. My RDS CPU dropped from 40% to 4%.
3 Guardrails You MUST Add
DLQ: Without it, one bad message can retry forever and cost you $$$.
Idempotency: SQS delivers at least once. Store order_id in DynamoDB with condition attribute_not_exists to avoid double email.
Visibility Timeout: Set to 6x your Lambda timeout. If Lambda takes 10s, set the timeout to 60s. Otherwise the message becomes visible again mid-processing.
DLQ: Without it, one bad message can retry forever and cost you $$$.
Idempotency: SQS delivers at least once. Store order_id in DynamoDB with condition attribute_not_exists to avoid double email.
Visibility Timeout: Set to 6x your Lambda timeout. If Lambda takes 10s, set the timeout to 60s. Otherwise the message becomes visible again mid-processing.
Built and tested on my own account. No secrets or account IDs shared. If you are still using cron to poll your DB, stop hit Like, and I’ll share the SAM template for SQS + Lambda + DLQ setup.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article