AWS Builder Center

Builder Center

Building Event-Driven Microservices with AWS Lambda and Amazon SQS

Learn how to decouple cloud microservices and build resilient, event-driven architectures using AWS Lambda, Amazon Simple Queue Service (SQS), and dead-letter queues (DLQs) to reliably handle traffic spikes without data loss.

In modern cloud architectures, direct synchronous calls between microservices can lead to tight coupling, cascading failures, and bottlenecks during traffic spikes. By combining Amazon Simple Queue Service (SQS) with AWS Lambda, you can build asynchronous, event-driven workflows that process messages reliably at scale.
This article covers how to decouple microservices using an SQS queue and an AWS Lambda consumer.

Why Pair AWS Lambda with Amazon SQS?

Using Amazon SQS as a buffer in front of an AWS Lambda function offers several core architectural advantages:
  • Traffic Smoothing: SQS holds incoming spikes in traffic, allowing Lambda to poll and process messages at a controlled rate.
  • Fault Tolerance: If a consumer fails to process a message, SQS automatically retries it or offloads it to a Dead-Letter Queue (DLQ).
  • Decoupled Components: Producers and consumers scale independently without needing direct service-to-service dependencies.

Step-by-Step Implementation

Step 1: Create the Amazon SQS Queue

  1. Open the Amazon SQS Console and click Create queue.
  2. Select Standard as the queue type.
  3. Name your queue OrderProcessingQueue.
  4. Leave default visibility timeouts and retention settings enabled, then click Create queue.

Step 2: Configure a Dead-Letter Queue (DLQ)

To capture failed messages and prevent infinite retry loops:
  1. Create a second standard queue named OrderProcessingQueue-DLQ.
  2. Edit OrderProcessingQueue, enable the Dead-letter queue section, and select OrderProcessingQueue-DLQ.
  3. Set the Maximum receives threshold to 3.

Step 3: Create the AWS Lambda Consumer

  1. Navigate to the AWS Lambda Console and click Create function.
  2. Choose Author from scratch, set the runtime (e.g., Python or Node.js), and assign an IAM role with the AWSLambdaSQSQueueExecutionRole policy.
  3. Under Add trigger, select SQS, pick OrderProcessingQueue, and configure your desired batch size (e.g., 10 messages).

Code Example: Processing SQS Messages in Python

Here is a minimal Lambda handler function in Python that parses incoming SQS batch records:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import json
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
batch_item_failures = []

for record in event.get('Records', []):
try:
message_body = json.loads(record['body'])
logger.info(f"Processing order ID: {message_body.get('order_id')}")

# Place your business processing logic here

except Exception as e:
logger.error(f"Error processing message {record['messageId']}: {str(e)}")
# Report individual item failure back to SQS
batch_item_failures.append({"itemIdentifier": record['messageId']})

return {"batchItemFailures": batch_item_failures}
# aws-lambda# amazon-sqs# serverless# microservices# event-driven
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article