AWS Builder Center
Stop Processing Everything at Once: Build a Reliable Queue with Amazon SQS and AWS Lambda

Stop Processing Everything at Once: Build a Reliable Queue with Amazon SQS and AWS Lambda

Your API doesn't need to handle every task before responding. Learn how to use Amazon SQS and AWS Lambda to process background jobs asynchronously, with a practical TypeScript example, simple architecture, and step-by-step testing.

Full Stack Developer | Backend & AWS Specialist | Node.js | TypeScript | Serverless | REST APIs
You've built an API using AWS Lambda and API Gateway.
A user sends a request, your Lambda processes it, and everything works.
Now imagine your application needs to do a little more.
When a customer places an order, your backend needs to save the order, send a confirmation email, and notify the warehouse.
Should the customer wait for all those tasks to finish before receiving a response?
Not necessarily.
Some work needs to happen immediately. Other work can happen in the background.
This is where Amazon Simple Queue Service (SQS) becomes useful.
In this article, we'll build a small example using Amazon SQS, AWS Lambda, Node.js, and TypeScript. By the end, you'll understand how to send a message to a queue and process it asynchronously.

1. Why Process Tasks in the Background?

Let's start with a simple scenario.
A customer places an order on an e-commerce website.
Your backend needs to:
  • Accept the order.
  • Save the order details.
  • Send a confirmation email.
  • Notify the warehouse.
Saving the order is important before confirming that it was accepted.
But does the customer need to wait for the confirmation email to be sent?
No.
Instead, your API can save the order, add a message to a queue, and return a response. Another Lambda function can process the notification later.
This approach is called asynchronous processing.
It helps separate the work that must happen during an API request from the work that can happen afterward.

2. What Is Amazon SQS?

Amazon SQS is a managed message queue service from AWS.
Think of it as a waiting area for tasks.
One part of your application sends a message to the queue. Another part receives that message and processes it.
In AWS terminology:
  • Producer: The application that sends messages.
  • Queue: The service that stores messages until they are processed.
  • Consumer: The application that processes messages.
For our example, we'll use two Lambda functions.
The first Lambda receives an API request and sends a message to SQS.
The second Lambda processes that message in the background.
Here's the architecture:
Customer
|
v
API Gateway
|
v
Order Lambda (Producer)
|
| Send message
v
Amazon SQS
|
| Trigger
v
Worker Lambda (Consumer)
|
v
Process order notification
One important detail: SQS doesn't make every operation faster. It allows tasks that don't need to finish during the HTTP request to run separately.

3. Create an SQS Queue

Let's create our first queue.
  1. Open the AWS Management Console.
  1. Search for Amazon SQS.
  1. Select Create queue.
  1. Choose Standard.
  1. Enter  order-notifications-queue  as the queue name.
  1. Keep the other settings at their defaults for this initial exercise and create the queue.
Why choose a Standard queue?
Standard queues support high throughput and at-least-once message delivery. That means a message can occasionally be delivered more than once, so our consumer must be designed to handle duplicates.
After creating the queue, copy its Queue URL.
We'll use that URL in our producer Lambda.

4. Send a Message to SQS Using TypeScript

Now let's create the Lambda function that receives an order request and sends a notification message to SQS.
We'll use AWS SDK for JavaScript v3.
Install the required packages in your Node.js project:
yarn add @aws-sdk/client-sqs
yarn add -D typescript @types/aws-lambda @types/node
Create a file named  producer.ts .
import type { APIGatewayProxyEventV2 } from "aws-lambda";
export const handler = async (
event: APIGatewayProxyEventV2
) => {
// Your Lambda logic
};

What's happening here?

When API Gateway sends a request to our Lambda:
  1. We read the  orderId  from the request body.
  1. We validate that an order ID was provided.
  1. We send a message to the SQS queue.
  1. We return HTTP  202 Accepted .
The  202  response means the notification task has been accepted for asynchronous processing. It doesn't mean the notification has already been completed.
For simplicity, this example focuses on queuing notifications. In a real order API, you would also validate the customer, authenticate the request, and save the order before confirming it.
Important: This example assumes API Gateway HTTP API payload format 2.0. You'll also need to compile your TypeScript into JavaScript before deploying it to a supported Node.js Lambda runtime.

5. Create the Worker Lambda

Our message is now sitting in the queue.
But who processes it?
That's the job of our second Lambda function.
Create a file named  worker.ts .
import type { SQSEvent } from "aws-lambda";
export const handler = async (
event: SQSEvent
): Promise<void> => {
for (const record of event.Records) {
const message = JSON.parse(record.body);
console.log("Processing message:", message);
}
};
This function reads messages received from SQS and logs their details.
For now, we're using  console.log()  to keep the example simple.
Later, you could connect Amazon SES or another notification provider to send actual emails.
Note: The worker doesn't send an email yet. It only demonstrates how SQS messages are consumed.

6. Connect SQS to the Worker Lambda

Now we need to tell AWS which Lambda should process messages from our queue.
In the AWS Console:
  1. Open your Worker Lambda function.
  1. Select Add trigger.
  1. Choose SQS.
  1. Select  order-notifications-queue .
  1. Set the batch size to  1  for this exercise.
  1. Enable the trigger.
AWS Lambda uses an event source mapping to poll the SQS queue and invoke your worker when messages are available.
For this setup, make sure:
  • The producer Lambda has permission to use  sqs:SendMessage  on the queue.
  • The worker Lambda execution role has the required SQS consumer permissions.
  • The queue visibility timeout is longer than the worker's execution timeout. AWS recommends at least six times the Lambda timeout when configuring an SQS event source mapping.
You can configure these permissions and resources using the AWS Console, CDK, or CloudFormation.

7. Test the Complete Flow

Now comes the interesting part.
Let's test our setup.
First, create an API Gateway HTTP API route connected to the producer Lambda.
For example:
 POST /orders 
Set the producer Lambda environment variable  QUEUE_URL  to your queue URL.
Then open Postman and send this request:
POST https://YOUR_API_URL/orders
Content-Type: application/json
Request body:
{
"orderId": "ORDER-1001"
}
If the producer successfully sends the message to SQS, you should receive a response similar to:
{
"message": "Notification queued",
"orderId": "ORDER-1001"
}
The HTTP status should be  202 Accepted .
Now open Amazon CloudWatch Logs and check the Worker Lambda log group.
You should find a log entry similar to:
Processing notification
orderId: ORDER-1001
action: SEND_CONFIRMATION
That's it.
Your API has accepted a request, sent a message to SQS, and allowed another Lambda to process it asynchronously.
The API doesn't need to wait for the worker to finish.

8. What Happens When Processing Fails?

There's one more thing worth understanding before using this pattern in a real application.
SQS Standard queues provide at-least-once delivery.
That means your worker might receive the same message more than once.
If the worker fails while processing a message, Lambda can retry it after the message becomes visible again.
For production workloads, consider:
  • Dead-letter queues (DLQs): Store messages that repeatedly fail processing.
  • Idempotency: Prevent duplicate messages from repeating business operations.
  • Visibility timeout: Give the worker enough time to process a message.
  • Monitoring: Track queue depth, message age, and failed processing.
In our small example, the worker only logs messages, so duplicate processing isn't a serious problem.
But imagine the worker sends payments or updates inventory. Processing the same event twice could have real consequences.
Also, because our worker processes a batch without catching individual errors, an exception causes the batch to fail. With a batch size of one, that's easier to understand while learning. For larger batches, partial batch responses are worth exploring.

When Should You Use SQS?

Amazon SQS is useful when you have tasks that can happen independently of the original API request.
Some common examples include:
  • Sending email notifications.
  • Processing uploaded files.
  • Generating reports.
  • Handling background jobs.
  • Passing tasks between microservices.
However, not every application needs a queue.
If an operation must finish before you can safely respond to the user, moving it to SQS might not be appropriate.
Adding SQS also introduces another component to configure, monitor, and maintain.
Use it when asynchronous processing solves a real problem, not just because it's available.

Final Thoughts

One thing I like about serverless architecture is that you can keep different parts of an application focused on specific responsibilities.
With Amazon SQS and AWS Lambda, your API can accept a request while a separate worker handles background processing.
In this example, we built a simple flow using:
API Gateway → Producer Lambda → Amazon SQS → Worker Lambda
It's a small example, but the same pattern can be useful in larger applications.
The code here demonstrates the basic flow. A production implementation would also need stronger validation, reliable order persistence, duplicate handling, failure recovery, and monitoring.
If you're learning AWS serverless development, try building this example yourself. Once it's working, a good next step is to add a dead-letter queue and experiment with failed messages.
The goal isn't to process everything at once. It's to process each task at the right time, in the right place.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article