AWS Builder Center

Building a Serverless Image Processing Pipeline on AWS:

In today’s fast-paced world, businesses are looking for efficient ways to process and analyze images in real-time. AWS provides a powerful serverless solution for image processing using services like Lambda, S3, and Rekognition. In this tutorial, we’ll guide you through the steps to create a serverless image processing pipeline.

In today’s fast-paced world, businesses are looking for efficient ways to process and analyze images in real-time. AWS provides a powerful serverless solution for image processing using services like Lambda, S3, and Rekognition. In this tutorial, we’ll guide you through the steps to create a serverless image processing pipeline.
“Now we See the step by step guide to create this POC which will detect for a specific text in the image uploaded to S3 and saves the result to dynamoDb as well as sends the notification to all the subscribers to the SNS topic:”


Step 1: Set Up an S3 Bucket

1.1. Sign in to AWS Console:

1.2. Access S3 Service:

  • In the AWS Management Console, search for “S3” or navigate to the “Storage” section and select “S3.”

1.3. Create a New Bucket:

  • Click the “Create bucket” button.

1.4. Configure Bucket Settings:

  • Fill in the following details:
  • Bucket name: Choose a globally unique name for your bucket.
  • Region: Choose the AWS region for your bucket.
  • Block all public access: Uncheck this option if you want your bucket to be public (make sure you understand the security implications).
  • Click “Next.”
  • Create the S3 bucket


Step 2: Create Lambda Functions

2.1. Access Lambda Service:

  • In the AWS Management Console, search for “Lambda” or navigate to the “Compute” section and select “Lambda.”

2.2. Create a New Lambda Function:

  • Click the “Create function” button.

2.3. Configure Function Basics:

  • Fill in the following details:
  • Function name: Enter a name for your Lambda function.
  • Runtime: Choose the runtime. For example, “Node.js 14.x” or “Python 3.8.”
  • Role: Choose “Choose an existing role” and select a role with the necessary permissions (or create a new role).
  • Click “Create function.”


2.4. Configure Function Trigger:

  • In the Lambda function’s configuration, add a trigger by clicking “Add trigger.”
  • Choose “S3” as the trigger source.
  • Select the S3 bucket you created in Step 1.
  • Set the event type to “Object Created (All).” This means the Lambda function will be triggered whenever a new object (image) is created in the S3 bucket.
  • Click “Add” to add the trigger.

2.5. Write Lambda Function Code:

  • Scroll down to the “Function code” section.
  • Below mentioned code is only for starting we will modufy this code in next steps
const AWS = require(‘aws-sdk’);
const rekognition = new AWS.Rekognition();
exports.handler = async (event) => {
const bucket = event.Records[0].s3.bucket.name;
const key = event.Records[0].s3.object.key;
console.log(`Image uploaded: ${key} in ${bucket}`);
// Add logic to call Rekognition and store results in DynamoDB
return {
statusCode: 200,
body: JSON.stringify(‘Image processing complete’),
};
};
ss


Step 3: Taking Amazon Recognition into picture

Below are the updated lambda code after taking Amazon recognition into picture which will detect any specific text in the uploaded Image and sends the notification to all the subscribers.
const AWS = require(‘aws-sdk’);
const rekognition = new AWS.Rekognition();
const dynamodb = new AWS.DynamoDB.DocumentClient();
const sns = new AWS.SNS();
exports.handler = async (event) => {
const bucket = event.Records[0].s3.bucket.name;
const key = event.Records[0].s3.object.key;
console.log(`Image uploaded: ${key} in ${bucket}`);
// Call Rekognition to detect text
const params = {
Image: {
S3Object: {
Bucket: bucket,
Name: key,
},
},
};
try {
const result = await rekognition.detectText(params).promise();
console.log(‘Text detected:’, result.TextDetections);
// Store results in DynamoDB
// Check conditions for sending notifications

return {
statusCode: 200,
body: JSON.stringify(‘Image processing complete’),
};
};

Step 4: Set Up DynamoDB

4.1. Access DynamoDB Service:

  • In the AWS Management Console, search for “DynamoDB” or navigate to the “Database” section and select “DynamoDB.”

4.2. Create a New DynamoDB Table:

  • Click the “Create table” button.

4.3. Configure Table Settings:

  • Fill in the following details:
  • Table name: Enter a name for your DynamoDB table.
  • Primary key: Enter a primary key for your table. For example, “ImageId” as the partition key.
  • Table settings: Adjust the read and write capacity settings based on your expected workload.
  • Click “Create.”

4.4. Add Additional Indexes (Optional):

  • Depending on your query requirements, you can add global secondary indexes to your table.
  • In the “Table details” tab, click on the “Indexes” tab, and then click “Create index.”

4.5. Update Lambda Function Code:

  • Update your Lambda function code to include logic for storing the results in DynamoDB.
const AWS = require(‘aws-sdk’);
const rekognition = new AWS.Rekognition();
const dynamodb = new AWS.DynamoDB.DocumentClient();
const sns = new AWS.SNS();
exports.handler = async (event) => {
const bucket = event.Records[0].s3.bucket.name;
const key = event.Records[0].s3.object.key;
console.log(`Image uploaded: ${key} in ${bucket}`);
// Call Rekognition to detect text
const params = {
Image: {
S3Object: {
Bucket: bucket,
Name: key,
},
},
};
try {
const result = await rekognition.detectText(params).promise();
console.log(‘Text detected:’, result.TextDetections);
// Store results in DynamoDB
const dynamoParams = {
TableName: ‘YourDynamoDBTableName’,
Item: {
ImageId: key, // Use a unique identifier for the image
TextDetections: result.TextDetections,
// Add other attributes as needed
},
};
await dynamodb.put(dynamoParams).promise();
console.log(‘Results stored in DynamoDB’);
// Check conditions for sending notifications
return {
statusCode: 200,
body: JSON.stringify(‘Image processing complete’),
};
};


Step 5: Implement SNS Notifications

5.1. Access SNS Service:

  • In the AWS Management Console, search for “SNS” or navigate to the “Application Integration” section and select “Simple Notification Service (SNS).”

5.2. Create a New SNS Topic:

  • Click the “Create topic” button.

5.3. Configure Topic Details:

  • Fill in the following details:
  • Name: Enter a name for your SNS topic.
  • Display name: (Optional) Enter a display name for your topic.
  • Click “Create topic.”
SS


5.4. Create a Subscription:

  • After creating the topic, click on the “Create subscription” button.
  • Choose the protocol for receiving notifications. For example, choose “Email” and enter your email address.
  • Click “Create subscription.”
  • Confirm the subscription by clicking the link sent to your email.

5.5. Update Lambda Function Code:

  • Update your Lambda function code to include logic for publishing messages to the SNS topic based on specific condition
const AWS = require(‘aws-sdk’);
const rekognition = new AWS.Rekognition();
const dynamodb = new AWS.DynamoDB.DocumentClient();
const sns = new AWS.SNS();
exports.handler = async (event) => {
const bucket = event.Records[0].s3.bucket.name;
const key = event.Records[0].s3.object.key;
console.log(`Image uploaded: ${key} in ${bucket}`);
// Call Rekognition to detect text
const params = {
Image: {
S3Object: {
Bucket: bucket,
Name: key,
},
},
};
try {
const result = await rekognition.detectText(params).promise();
console.log(‘Text detected:’, result.TextDetections);
// Store results in DynamoDB
const dynamoParams = {
TableName: ‘YourDynamoDBTableName’,
Item: {
ImageId: key, // Use a unique identifier for the image
TextDetections: result.TextDetections,
// Add other attributes as needed
},
};
await dynamodb.put(dynamoParams).promise();
console.log(‘Results stored in DynamoDB’);
// Check conditions for sending notifications
if (result.TextDetections.some(text => text.Type === ‘LINE’ && text.DetectedText.includes(‘YourSpecificText’))) {
const snsParams = {
Message: `Image with specific text detected: ${key}`,
Subject: ‘Image Processing Alert’,
TopicArn: ‘YourSNSTopicARN’,
};
await sns.publish(snsParams).promise();
console.log(‘Notification sent’);
}
} catch (error) {
console.error(‘Error detecting text:’, error);
}
return {
statusCode: 200,
body: JSON.stringify(‘Image processing complete’),
};
};
SS


Step 6: Test the Serverless Image Processing Pipeline

6.1. Upload Test Images to S3 Bucket:

  • Go to the S3 bucket created in Step 1.
  • Click on the “Upload” button and upload one or more test images to the bucket.
SS

Check your subscribed emil you have got one email like this.
SS


6.3. Check DynamoDB for Results:

  • Open the DynamoDB Console.
  • Click on the DynamoDB table created in Step 4.
  • Check the “Items” tab to verify that the results of image analysis are correctly stored in DynamoDB.
SS

We successfully Implemented one POC in which we setted up a serverless architecture using S3, Lambda, DynamoDb, Recognition , and SNS to detected the Text in any image uploaded to a S3 bucket.
Hope this will help . Thank you!!!
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article