Building a Serverless REST API on AWS - Part 3: Security, Monitoring, Performance, and Production Readiness
Harden your serverless API with security, IAM, monitoring, logging, performance tuning, deployment, and cost optimization.
In Part 1, we designed the architecture and connected Amazon API Gateway, AWS Lambda, and Amazon DynamoDB.
In Part 2, we implemented our CRUD operations and turned that architecture into a working API.
Now we have something functional.
But functional and production-ready are not the same thing.
A production API needs to answer questions such as:
- Who is allowed to call it?
- What happens when a client sends invalid data?
- Can one Lambda function access resources it does not need?
- How do we diagnose a failed request?
- What happens during a sudden traffic spike?
- Can we recover accidentally deleted DynamoDB data?
- How do we prevent unexpected serverless costs?
- How do we deploy changes safely?
That is what we will address in Part 3.
Our Starting Architecture
The API from the previous parts looks approximately like this:
Client | vAmazon API Gateway | vAWS Lambda | vAmazon DynamoDB
This is already a powerful architecture.
However, our production version will look conceptually more like this:
+----------------------+ | CloudWatch | | Logs / Metrics / | | Alarms | +----------^-----------+ | |Client | | | v |+----------------------+ || API Gateway HTTP API |------------+| || CORS || JWT Authorization || Access Logging |+----------+-----------+ | v+----------------------+| AWS Lambda || || Input Validation || Structured Logging || Error Handling || Least Privilege IAM || Concurrency Controls |+----------+-----------+ | v+----------------------+| Amazon DynamoDB || || On-Demand Capacity || Conditional Writes || PITR / Backups || Access Patterns |+----------------------+
The number of services has barely changed.
What changes is how carefully we configure them.
1. Start with Least-Privilege IAM
One of the biggest mistakes in early AWS projects is giving Lambda permissions such as:
{ "Effect": "Allow", "Action": "*", "Resource": "*"}
It certainly makes permission errors disappear.
It also gives the function far more power than it should have.
In our API, Lambda only needs access to the DynamoDB table used by our application.
A simplified policy could look like this:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ItemsTableAccess", "Effect": "Allow", "Action": [ "dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:UpdateItem", "dynamodb:DeleteItem" ], "Resource": "arn:aws:dynamodb:REGION:ACCOUNT_ID:table/Items" } ]}
Replace:
REGIONACCOUNT_IDItems
with your actual values.
If your API performs a
Query , add:dynamodb:Query
If the API still has a
Scan operation, it will also require:dynamodb:Scan
However, I would avoid designing a production API around large table scans.
As the table grows, querying data according to known access patterns becomes much more scalable.
An Even Better Design
Suppose we created four separate functions:
CreateItemFunctionGetItemFunctionUpdateItemFunctionDeleteItemFunction
There is no reason for
GetItemFunction to have permission to delete records.Its policy could contain only:
{ "Effect": "Allow", "Action": [ "dynamodb:GetItem" ], "Resource": "arn:aws:dynamodb:REGION:ACCOUNT_ID:table/Items"}
Similarly, the delete function only needs
DeleteItem .This reduces the blast radius if one function is ever compromised.
Least privilege should be applied not just at the application level but, where practical, at the individual function level.
2. Move Configuration Out of the Code
Our Lambda function should not contain hard-coded resource names such as:
table = dynamodb.Table("Items-Production-Europe")
A better approach is to configure the table through an environment variable.
import osimport boto3
TABLE_NAME = os.environ["TABLE_NAME"]
dynamodb = boto3.resource("dynamodb")table = dynamodb.Table(TABLE_NAME)
Our environments can now have different tables:
Development -> Items-devStaging -> Items-stageProduction -> Items-prod
while running exactly the same application code.
AWS recommends environment variables for operational configuration rather than hard-coding these values into Lambda functions.
Do Not Store Secrets There Just Because You Can
A table name is configuration.
A database password, API credential, or external service token is a secret.
If our application later requires sensitive credentials, AWS recommends services such as AWS Secrets Manager rather than storing those secrets directly as Lambda environment variables.
Our current DynamoDB integration does not require a database password because authentication happens through the Lambda execution role.
That is one of the nice characteristics of using AWS-native services together.
3. Validate Every Request
Never assume that the client sends valid JSON.
Consider this request:
{ "name": "", "price": -100}
Technically, it is JSON.
That does not mean it should enter our database.
Our application should validate input before performing a DynamoDB operation.
For example:
def validate_item(data, partial=False): errors = []
if not partial or "name" in data: name = data.get("name")
if not isinstance(name, str) or not name.strip(): errors.append("name must be a non-empty string")
elif len(name) > 120: errors.append("name cannot exceed 120 characters")
if not partial or "price" in data: price = data.get("price")
if not isinstance(price, (int, float)): errors.append("price must be a number")
elif price < 0: errors.append("price cannot be negative")
return errors
Then our create operation can perform validation before calling DynamoDB:
data = json.loads(event.get("body") or "{}")
errors = validate_item(data)
if errors: return response( 400, { "message": "Validation failed", "errors": errors } )
The important principle is:
Validate before persistence.
Do not let malformed data enter DynamoDB and attempt to fix it later.
4. Return Consistent HTTP Responses
A client should not have to guess what an error means.
Our API should follow a predictable error model.
For example:
{ "error": { "code": "VALIDATION_ERROR", "message": "The request contains invalid fields." }}
We could create one reusable response function:
import jsonfrom decimal import Decimal
def json_default(value): if isinstance(value, Decimal): return float(value)
raise TypeError
def response(status_code, body): return { "statusCode": status_code, "headers": { "content-type": "application/json" }, "body": json.dumps(body, default=json_default) }
Then:
return response( 404, { "error": { "code": "ITEM_NOT_FOUND", "message": "The requested item does not exist." } })
A useful status-code strategy for this API is:
200 -> Successful read or update201 -> Resource created204 -> Resource successfully deleted
400 -> Invalid request401 -> Authentication required403 -> Authenticated but not allowed404 -> Resource does not exist409 -> Resource conflict429 -> Too many requests500 -> Unexpected server error
Most importantly, avoid exposing internal exceptions to API clients.
This is bad:
{ "error": "botocore.exceptions.ClientError at line 183..."}
The client does not need your stack trace.
Return a safe message to the client and put the detailed exception in your logs.
5. Use Conditional Writes
There is a subtle problem with this code:
table.put_item(Item=item)
What happens if an item with that key already exists?
Depending on our design, we may unintentionally overwrite it.
For a create endpoint, we can prevent that using a DynamoDB condition:
table.put_item( Item=item, ConditionExpression="attribute_not_exists(itemId)")
Now the operation succeeds only if the item does not already exist.
We can catch the conflict:
from botocore.exceptions import ClientError
try: table.put_item( Item=item, ConditionExpression="attribute_not_exists(itemId)" )
except ClientError as exc: if exc.response["Error"]["Code"] == "ConditionalCheckFailedException": return response( 409, { "error": { "code": "ITEM_ALREADY_EXISTS", "message": "An item with this ID already exists." } } )
raise
A similar concept can be used when updating.
Instead of allowing
UpdateItem to silently create a record that does not exist, we can require:ConditionExpression="attribute_exists(itemId)"
These small details make the behavior of the API much more predictable.
6. Think About Idempotency
Imagine a user presses Create.
The request reaches our API and DynamoDB successfully stores the item.
But before the HTTP response reaches the client, the user's connection fails.
The client does not know whether the request succeeded.
It retries.
Without additional protection, we could create the same logical operation twice.
For important create or payment-style operations, clients can provide something such as:
Idempotency-Key: 8a24a7e6-...
The backend records that key and ensures that repeating the same operation does not create another result.
AWS Powertools for Lambda includes utilities for common production concerns including idempotency, structured logging, metrics, tracing, and input validation.
For a small learning API this may be unnecessary.
For a real production system, idempotency is something you should consciously evaluate rather than ignore.
7. Configure CORS Properly
If a browser frontend calls our API, we need to configure Cross-Origin Resource Sharing.
During development, it is tempting to configure:
Allowed origin: *
For a production application, it is generally better to specify the frontend that actually needs access.
For example:
https://app.example.com
A configuration might conceptually look like:
{ "AllowOrigins": [ "https://app.example.com" ], "AllowMethods": [ "GET", "POST", "PUT", "DELETE", "OPTIONS" ], "AllowHeaders": [ "content-type", "authorization" ]}
For an API Gateway HTTP API, CORS can be configured directly on API Gateway. AWS documents support for allowed origins, methods, headers, credentials, exposed headers, and cache duration.
That keeps CORS policy centralized at the API layer instead of spreading it throughout every Lambda response.
8. Protect the API with JWT Authorization
At this stage our API may still be publicly callable.
That might be intentional for some endpoints.
It probably should not be true for all of them.
API Gateway HTTP APIs support JWT authorizers.
This means API Gateway can validate a JSON Web Token before Lambda is invoked. It can validate properties such as the token signature, issuer, audience, and authorization scopes.
A common architecture becomes:
User | vIdentity Provider | | receives JWT vClient | | Authorization: Bearer <token> vAPI Gateway | | validate token | +------ invalid ------> 401 / 403 | +------ valid | vLambda
The significant point here is that invalid requests can be rejected before our business logic executes.
We can also protect routes differently.
For example:
GET /itemsRequired scope:items:read
POST /itemsRequired scope:items:write
PUT /items/{id}Required scope:items:write
DELETE /items/{id}Required scope:items:delete
API Gateway supports configuring authorization scopes for HTTP API routes.
Your identity provider could be Amazon Cognito or another compatible OIDC/OAuth 2.0 provider.
9. Introduce Structured Logging
This line is useful during development:
print("Something happened")
It is considerably less useful when you have thousands of requests.
Instead, logs should contain structured information.
For example:
{ "level": "INFO", "service": "items-api", "operation": "create-item", "itemId": "7c813e13", "requestId": "abc123", "message": "Item created successfully"}
Structured logs allow us to search fields rather than arbitrary text.
AWS Lambda currently supports structured JSON logging for supported managed runtimes, and AWS recommends structured logging as a Lambda best practice.
With Python's logging library:
import logging
logger = logging.getLogger()logger.setLevel(logging.INFO)
Then:
logger.info( "Item created", extra={ "itemId": item_id })
For larger projects, AWS Powertools can make structured logging and correlation considerably easier.
10. Never Log Sensitive Data
Logging everything sounds useful until the logs contain:
Authorization headersJWTsPasswordsAPI keysCredit card informationPersonal data
Do not blindly log:
logger.info(event)
A complete API Gateway event can contain headers and potentially sensitive client information.
Instead, log only the fields required for operating and troubleshooting the service.
A better request log might include:
requestIdroutemethodstatusdurationitemIduserId
without recording secrets or tokens.
11. Enable API Gateway Access Logs
Lambda logs tell us what happens inside our application.
API Gateway access logs tell us what happens at the HTTP boundary.
For an HTTP API, API Gateway can send access logs to CloudWatch Logs. AWS supports context variables including request ID, route, status, response latency, integration latency, and integration errors.
A useful JSON log format could be:
{ "requestId": "$context.requestId", "routeKey": "$context.routeKey", "status": "$context.status", "responseLatency": "$context.responseLatency", "integrationLatency": "$context.integrationLatency", "integrationError": "$context.integrationErrorMessage"}
Now imagine a user reports:
My request failed at 14:32.
Instead of randomly searching through Lambda logs, we can start with API Gateway:
requestId | vAPI Gateway access log | vLambda invocation | vApplication logs
Request IDs become extremely useful for correlating activity across the system.
12. Set CloudWatch Log Retention
There is another detail that is easy to overlook.
Lambda logs do not simply disappear when you delete a function. AWS specifically notes that log groups are not automatically removed with the function.
For development environments, keeping logs forever usually provides little benefit.
Set a retention policy appropriate for your requirements.
For example:
Development -> 7 daysStaging -> 14 daysProduction -> 30 / 90+ days
The correct value depends on operational, security, and compliance requirements.
The important point is to make retention deliberate.
Otherwise an application can quietly accumulate CloudWatch Logs and associated costs for years.
13. Monitor Metrics, Not Just Logs
Logs answer:
What happened?
Metrics answer:
Is something going wrong right now?
API Gateway HTTP APIs automatically publish CloudWatch metrics including:
Count4xx5xxLatencyIntegrationLatencyDataProcessed
AWS also supports optional detailed route-level metrics.
Lambda automatically provides operational metrics as well, including invocation activity, duration, errors, concurrency, and throttling information.
A useful dashboard for this application might contain:
API REQUESTSCount4xx responses5xx responses
LATENCYAPI Gateway LatencyIntegrationLatencyLambda Duration
LAMBDAInvocationsErrorsThrottlesConcurrentExecutions
DYNAMODBRead activityWrite activityThrottling
This gives us an immediate picture of the health of the application.
14. Add CloudWatch Alarms
Dashboards are useful when somebody is looking at them.
Alarms are useful when nobody is.
Examples include:
API 5xx errors unexpectedly increaseLambda error count becomes non-zeroLambda throttling occursLatency exceeds the expected baselineDynamoDB begins throttling requests
The threshold should depend on the application.
For a personal project, one Lambda error may be worth investigating.
For an application processing millions of requests, a better alarm might use an error percentage rather than absolute count.
Production monitoring should reflect your service-level expectations rather than arbitrary numbers copied from a tutorial.
15. Reuse AWS SDK Clients
Consider this code:
def lambda_handler(event, context): dynamodb = boto3.resource("dynamodb") table = dynamodb.Table("Items")
It creates the resource inside the handler for every invocation.
A better Lambda pattern is:
import boto3import os
dynamodb = boto3.resource("dynamodb")table = dynamodb.Table(os.environ["TABLE_NAME"])
def lambda_handler(event, context): ...
Initialization outside the handler can be reused when Lambda reuses an execution environment.
AWS explicitly recommends initializing SDK clients and database connections outside the handler when appropriate so that subsequent invocations can reuse them.
This is one of the simplest Lambda optimizations available.
16. Control Lambda Concurrency
One of Lambda's greatest strengths is its ability to scale.
That can also become a problem.
Suppose traffic suddenly jumps from:
10 requests/sec
to:
10,000 requests/sec
Serverless services can scale very rapidly.
Sometimes we do not want unlimited scaling.
Lambda reserved concurrency can set a maximum number of concurrent executions for a function. AWS specifically describes this as useful for protecting downstream systems from being overwhelmed.
For example:
Reserved concurrency: 100
would prevent that function from consuming unlimited account concurrency or placing unrestricted pressure on downstream dependencies.
Reserved concurrency itself does not incur an additional charge.
What About Provisioned Concurrency?
Provisioned concurrency keeps execution environments initialized and can reduce cold-start latency.
But it also costs money.
Do not enable it simply because it sounds more “production.”
Use it when your latency requirements justify it.
AWS notes that provisioned concurrency is particularly useful for latency-sensitive interactive workloads and incurs additional charges.
Measure first.
Optimize second.
17. DynamoDB Capacity Strategy
For this type of serverless API, DynamoDB on-demand capacity is an excellent starting point.
With on-demand mode, DynamoDB automatically manages throughput and charges based on the read and write requests performed.
AWS currently describes on-demand mode as the default and recommended throughput option for most workloads.
That makes it especially convenient for APIs with unpredictable traffic.
However, “serverless” does not mean “impossible to overload.”
Large and extremely sudden jumps beyond previous traffic levels can still require capacity considerations.
18. Add a DynamoDB Cost Guardrail
There is a useful production feature that is easy to miss.
For DynamoDB on-demand tables, we can configure maximum read and write throughput.
For example:
Maximum reads/sec: 5,000Maximum writes/sec: 2,000
Requests above the configured maximum can be throttled.
Why intentionally place a ceiling on a serverless database?
Because uncontrolled scaling can also mean uncontrolled spending.
AWS specifically positions maximum on-demand throughput as a way to improve cost predictability and protect against unexpected usage.
This gives us another guardrail against:
bugsrunaway scriptsunexpected loadabusive clientsaccidental loops
Again, the actual values should come from observed traffic rather than guesswork.
19. Design DynamoDB Around Access Patterns
A production DynamoDB design should start with questions such as:
How will I retrieve an item?How will I retrieve items belonging to one user?Will I list items by status?Will I sort by creation date?
Suppose our items contain:
{ "itemId": "item-123", "userId": "user-42", "name": "Mechanical Keyboard", "status": "ACTIVE", "createdAt": "2026-08-31T08:00:00Z"}
Getting one record using:
itemId
is straightforward.
But suppose we need:
GET /users/{userId}/items
A table scan followed by filtering is usually not the architecture we want.
Instead, we might create a Global Secondary Index based on:
userIdcreatedAt
which lets DynamoDB answer the access pattern efficiently with
Query .The important DynamoDB lesson is:
Design the table around the questions your application needs to ask.
Not:
Create a table first and figure out queries later.
20. Enable Point-in-Time Recovery
What happens if someone accidentally deletes data?
Or a deployment contains a bug that corrupts hundreds of records?
DynamoDB Point-in-Time Recovery provides continuous backups and supports restoring table data to a previous point in time.
AWS currently supports recovery points with per-second granularity within a configurable recovery period of up to 35 days.
One important behavior to understand is that restoring a table creates a new DynamoDB table rather than modifying the existing table in place.
For production data, PITR is a relatively simple protection that can save you from a very painful incident.
21. Separate Development, Staging, and Production
A common early architecture looks like:
One LambdaOne DynamoDB tableOne API
used for everything.
That means testing a new delete feature can accidentally delete production data.
A healthier setup is:
DEVAPI-devLambda-devItems-dev
STAGEAPI-stageLambda-stageItems-stage
PRODAPI-prodLambda-prodItems-prod
Each environment should be deployable independently.
For more mature systems, they may even live in separate AWS accounts.
The important principle is that experimenting in development should not put production resources at risk.
22. Deploy Infrastructure as Code
By Part 3, manually recreating everything through the console should no longer be our deployment strategy.
Our architecture should be defined as code.
Options include:
AWS SAMAWS CDKAWS CloudFormationTerraform
For example, the infrastructure definition should describe:
DynamoDB tableLambda functionsLambda IAM rolesAPI Gateway routesEnvironment variablesAuthorizersLog groupsAlarms
Now infrastructure changes can go through:
Git | vPull Request | vCode Review | vDeployment Pipeline
instead of:
Someone clicked something in the AWS console.
Infrastructure as Code makes environments more repeatable and makes architectural changes reviewable.
23. Test Failure Paths
Most tutorials test:
POST /items
and celebrate when the response is:
201 Created
Production testing needs to go much further.
We should test scenarios such as:
Valid create requestInvalid JSONMissing required fieldNegative priceUnknown item IDDuplicate item IDUnauthorized requestExpired JWTInsufficient authorization scopeMalformed path parameterDynamoDB failureLambda exceptionLarge payloadRapid repeated requests
The unhappy paths are often more important than the happy path.
A robust API is not one that works when everything is correct.
It is one that fails predictably when something is wrong.
24. Add Cost Visibility
Serverless architectures remove a lot of infrastructure management.
They do not remove the need to understand cost.
For this architecture, I would monitor at least:
API Gateway requestsLambda invocationsLambda durationLambda memory configurationDynamoDB reads and writesCloudWatch Logs ingestion/storageCloudWatch custom/detailed metricsData transfer
Detailed API Gateway route metrics, for example, can introduce additional CloudWatch charges.
CloudWatch logs, dashboards, alarms, and enhanced monitoring also have pricing implications beyond their free-tier allowances.
Production architecture is about balancing: not maximizing one of them while ignoring the others.
25. Our Final Architecture
After these improvements, the architecture is still recognizably serverless:
Authentication | vClient ---------> API Gateway HTTP API | | JWT authorization | CORS | Access logging | Throttling boundary | v AWS Lambda | | Input validation | Structured logging | Error handling | Least-privilege IAM | Conditional writes | Concurrency control | v Amazon DynamoDB | | On-demand capacity | Access-pattern design | Throughput guardrails | Point-in-time recovery | v Application Data
API Gateway --------+Lambda -------------+-----> Amazon CloudWatchDynamoDB ------------+ Logs Metrics Dashboards Alarms
There are still no EC2 instances.
There are no operating systems to patch.
There is no database server to maintain.
But that does not mean architecture disappeared.
Serverless removes much of the infrastructure management.
It does not remove software engineering.
Production Checklist
Before calling our API production-ready, I would verify:
SECURITY[ ] Lambda uses least-privilege IAM[ ] No credentials are stored in source code[ ] Sensitive values use an appropriate secrets service[ ] Protected routes require authorization[ ] Authorization scopes match route permissions[ ] CORS allows only required origins
APPLICATION[ ] Input is validated[ ] Errors use consistent HTTP responses[ ] Internal exceptions are not exposed[ ] Create/update operations use appropriate conditions[ ] Idempotency has been considered
OBSERVABILITY[ ] Lambda structured logging is enabled[ ] API Gateway access logging is enabled[ ] Sensitive information is excluded from logs[ ] Log retention is configured[ ] CloudWatch dashboard exists[ ] Important alarms exist
RELIABILITY[ ] DynamoDB PITR is enabled where appropriate[ ] Failure scenarios have been tested[ ] Lambda concurrency behavior is understood
PERFORMANCE[ ] AWS SDK clients are initialized outside handlers[ ] DynamoDB access patterns avoid unnecessary scans[ ] Lambda memory and duration have been measured[ ] Provisioned concurrency is used only when justified
COST[ ] CloudWatch logging volume is monitored[ ] DynamoDB capacity mode is intentional[ ] Throughput guardrails are considered[ ] Unexpected traffic scenarios are considered
DEPLOYMENT[ ] Infrastructure is defined as code[ ] Development and production are separated[ ] Changes are version controlled[ ] Deployment is repeatable
What We Built Across This Series
Over these three articles, we moved through three very different stages.
Part 1 - Architecture
We started with:
API Gateway |Lambda |DynamoDB
and learned how the AWS services communicate.
Part 2 - Application
We turned the architecture into a real CRUD API:
POST /itemsGET /itemsGET /items/{id}PUT /items/{id}DELETE /items/{id}
and implemented the application logic inside Lambda.
Part 3 - Production Engineering
Finally, we added the things that are easy to ignore when building a demo:
IAMAuthenticationValidationConditional writesCORSLoggingMetricsAlarmsConcurrencyDynamoDB optimizationBackupsInfrastructure as CodeCost controls
And that is probably the most important lesson from this series.
Final Thoughts
Building a serverless API is easy to demonstrate.
Building one that you can confidently operate is a much more interesting engineering problem.
API Gateway, Lambda, and DynamoDB handle a huge amount of infrastructure for us, but AWS cannot decide our application's security boundaries, data model, authorization rules, error handling strategy, monitoring requirements, or acceptable costs.
Those decisions still belong to us.
The biggest thing I learned while building this project is that serverless does not mean architecture-less.
In many ways, it makes good architecture even more important because each managed service becomes an explicit boundary in the system.
Once you start thinking about those boundaries-who can access them, how they fail, how they scale, and how you observe them-you move from simply using AWS services to actually designing systems on AWS.
And that is where serverless gets much more interesting.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article