AWS Lambda for Long-Running Workloads
Explore AWS Lambda Managed Instances for long-running workloads, including 90-minute timeouts, event processing, AI inference, and batch jobs.
Systems and Computer Engineering student and full-stack developer interested in cloud computing
AWS Lambda for Long-Running Workloads
AWS Lambda has traditionally been associated with short-lived, event-driven functions.
That model works extremely well for APIs, file processing, automation, and lightweight background jobs.
However, some workloads naturally take much longer to complete.
Examples include:
text
AI inferenceMedia transcodingLarge data processingBatch workloadsDocument processingLong-running integrations
On September 9, 2026, AWS announced support for a 90-minute function timeout for asynchronous and event source mapping invocations on Lambda Managed Instances, increasing the previous 15-minute limit by six times. AWS specifically highlighted data processing, media transcoding, AI inference, and batch workloads as use cases that can benefit from the change.
This changes the types of workloads developers can consider for Lambda.
The traditional Lambda model
A simplified event-driven application might look like:
text
Event│▼AWS Lambda│▼Result
For many applications, the function completes quickly:
text
Request│▼Lambda│▼Response
This is one of the reasons Lambda became such a popular building block for serverless architectures.
But not every workload is short.
Consider a large media transformation:
text
Upload Video│▼Process Video│├── Decode├── Transform├── Encode└── Store│▼Completed File
Depending on the file and processing requirements, the workload can take significantly longer than a traditional request-response operation.
What changed?
AWS Lambda Managed Instances now support a maximum function timeout of 90 minutes for asynchronous and event source mapping invocations. AWS describes this as a sixfold increase from the previous 15-minute limit.
Conceptually:
text
Previous───────────────0 min 15 minCurrent────────────────────────────────────────────0 min 90 min
This does not mean every Lambda function should run for 90 minutes.
The important change is that workloads previously constrained by the execution window can now be considered for Lambda Managed Instances without automatically moving to a completely different compute architecture.
What are Lambda Managed Instances?
Lambda Managed Instances allow Lambda functions to run on a managed fleet of compute resources while retaining Lambda's event-driven programming model.
The key idea is:
text
Developer│▼Lambda Function│▼Managed Compute│▼AWS
Instead of managing a traditional server fleet directly, developers continue working with Lambda while AWS manages the underlying execution environment.
This can be useful when the workload needs more control over compute characteristics than a conventional short-lived serverless invocation provides.
A long-running processing example
Imagine an application that receives large data files.
The workflow might be:
text
Client│▼Amazon S3│▼Event│▼Lambda Managed Instance│├── Read file├── Validate data├── Transform records├── Generate output└── Store results│▼Amazon S3
Previously, developers might have needed another compute option when a single processing operation could exceed Lambda's execution limit.
With the increased timeout, some of these workloads can remain within the Lambda ecosystem.
Long-running does not mean unlimited
There is an important distinction here.
A 90-minute timeout is a maximum execution window, not a recommendation.
Running a function for a long time can introduce other concerns:
text
Execution costFailure recoveryRetriesConcurrencyState managementExternal dependenciesObservability
A function that normally takes 30 seconds but occasionally runs for 40 minutes may require a very different architecture from one that consistently runs for 80 minutes.
The timeout is therefore an architectural capability, not a reason to ignore workload characteristics.
Asynchronous processing
Long-running workloads are often better suited to asynchronous architectures.
Instead of:
text
User│▼API│▼Long-running process│▼Response
we can use:
text
User│▼API│▼Create Job│▼Queue / Event│▼Lambda│▼Process
The user does not need to remain connected while the work is being completed.
For example:
text
POST /jobs
could return:
JSON
{"jobId": "job-001","status": "processing"}
The client can then check the status later:
text
GET /jobs/job-001
This pattern is generally more resilient for long-running work.
Event source mappings
Lambda can also process events from supported event sources.
A conceptual architecture might be:
text
Event Source│▼Event Source Mapping│▼Lambda Managed Instance│▼Processing
This model is particularly useful when a continuous stream of messages needs to be processed asynchronously.
Instead of requiring the producer to wait for the consumer, events can accumulate while processing continues.
AI inference
One of the areas highlighted by AWS for the extended timeout is AI inference.
Consider an AI pipeline:
text
Input Data│▼Preprocessing│▼Model Inference│▼Postprocessing│▼Result
A complex inference pipeline may involve:
text
Large filesMultiple model callsData transformationExternal servicesPostprocessing
Instead of requiring the user to keep an HTTP connection open, the application can treat inference as a background job.
For example:
text
User│▼Submit AI Job│▼Queue│▼Lambda│├── Load data├── Run inference├── Process output└── Store result│▼Notification
This can make the application easier to design and operate.
Media transcoding
Media workloads are another natural example.
A video processing pipeline may look like:
text
Original Video│▼Download / Read│▼Decode│▼Transform│▼Encode│▼Upload Output
Different resolutions and codecs can dramatically change processing time.
The ability to support longer-running execution means some media workflows can be implemented using Lambda Managed Instances without splitting the workload into many small functions purely because of the timeout constraint.
AWS explicitly lists media transcoding among the workloads that can benefit from the new timeout.
Batch processing
Batch workloads are another useful pattern.
Suppose an application needs to process:
text
10,000 documents
A naive approach might process everything inside one invocation.
That can be risky.
A better approach could be:
text
Input Files│▼Queue / Events│┌─────────┼─────────┐▼ ▼ ▼Worker A Worker B Worker C│ │ │└─────────┼─────────┘▼Results
Longer execution windows can help individual workers perform more substantial units of work, while queues and concurrency can still provide parallelism.
The architectural objective should be to balance:
text
Work per invocation+Parallelism+Failure isolation
What happens when a long-running job fails?
Long execution times increase the cost of failure.
Imagine a job that runs for 50 minutes and then crashes.
If the system automatically retries the entire operation, it could repeat almost all of the work.
This makes checkpointing and idempotency especially important.
A conceptual strategy is:
text
Start Job│▼Step 1 ✓│▼Step 2 ✓│▼Step 3 ✗│▼Resume / Retryfrom checkpoint
The implementation depends on the workload.
For example, a batch application may store completed units in DynamoDB or S3 so that a retry does not repeat work unnecessarily.
Idempotency still matters
A long-running function can still be retried.
Suppose an event represents:
text
process-file-123
The application should be able to determine whether the work has already been completed.
A conceptual implementation might look like:
text
Event ID│▼Already processed?│┌─┴─────┐Yes No│ │▼ ▼Skip Process│▼Mark complete
Without idempotency, retries can create:
text
Duplicate recordsDuplicate chargesRepeated notificationsCorrupted state
Longer execution windows do not eliminate these distributed-system problems.
Timeouts are only one part of reliability
A production workload should consider what happens at every stage.
For example:
text
Job│▼Processing│┌─────┴─────┐│ │Success Failure│ │▼ ▼Complete Retry│▼DLQ
For event-driven workloads, dead-letter queues and retry policies can provide an important safety mechanism.
The application should have a clear path for messages that repeatedly fail.
Observability becomes more important
A five-second function is relatively easy to debug.
A function that runs for 30 or 60 minutes requires much better visibility.
Useful telemetry can include:
text
Job IDInvocation IDStart timeProgressProcessing stageElapsed timeErrorsRetriesOutput location
For example:
text
JOB: 8F21A[██████████████░░░░░░] 70%Stage:TranscodingElapsed:38m 12sStatus:RUNNING
The exact implementation may use Amazon CloudWatch logs, metrics, and application-level progress tracking.
The important principle is that a long-running workload should not become an opaque process.
Cost considerations
Longer execution times naturally require more attention to cost.
A useful mental model is:
text
Cost=Execution×Memory / Compute×Number of Invocations
But real cost depends on the specific Lambda Managed Instances configuration and pricing model.
The main lesson is that a long-running architecture should be measured using actual workload characteristics instead of assuming that serverless is automatically cheaper.
For example:
text
Job A2 minutes × 1,000 executionsJob B40 minutes × 50 executions
The second workload has fewer invocations but potentially much larger execution time.
Cost analysis therefore needs to reflect the whole workload.
When Lambda is a good fit
The extended timeout makes Lambda Managed Instances more interesting for:
text
AI inferenceMedia processingBatch jobsLarge data transformationsAsynchronous workflowsEvent-driven processing
AWS specifically highlights these categories as examples that can benefit from the longer execution window.
However, a long-running job does not automatically belong on Lambda.
Other AWS compute services may still be more appropriate when the workload requires specialized infrastructure, persistent processes, advanced scheduling, or very specific control over the operating environment.
A practical architecture
Consider a document-processing platform.
Users upload documents to Amazon S3:
text
User│▼Amazon S3│▼Object Created Event│▼Lambda Managed Instance│├── Extract content├── Transform data├── Run AI processing├── Generate metadata└── Store results│▼Amazon S3 / Database
A notification can then inform the application:
text
Processing complete
The frontend never needs to remain connected for the entire processing period.
This makes the architecture asynchronous and easier to scale.
Designing around jobs
One of the most useful mental shifts is to stop thinking of long-running operations as requests.
Instead, think of them as jobs.
Rather than:
text
Request↓Wait↓Response
use:
text
Request↓Create Job↓Process Asynchronously↓Track Status↓Retrieve Result
This pattern works particularly well when the workload is unpredictable.
For example:
text
POST /jobs
returns:
JSON
{"jobId": "abc123","status": "queued"}
Then:
text
GET /jobs/abc123
might return:
JSON
{"jobId": "abc123","status": "processing","progress": 72}
Finally:
JSON
{"jobId": "abc123","status": "completed","result": "s3://..."}
This API design separates user interaction from backend execution.
The architectural trade-off
The new timeout does not eliminate the need for architecture.
It gives developers another option.
Before designing a workload, ask:
text
How long does the job run?How often does it run?Can it be parallelized?What happens when it fails?Can it be retried safely?Does it need persistent state?How will progress be monitored?What does it cost?
These questions are more important than simply asking:
"Can Lambda run this?"
Final thoughts
The increase to a 90-minute timeout for asynchronous and event source mapping invocations on Lambda Managed Instances expands the range of workloads that can be considered for Lambda. AWS specifically points to AI inference, media transcoding, data processing, and batch workloads as potential beneficiaries.
But the real architectural lesson goes beyond the timeout itself.
Long-running workloads need:
text
Asynchronous design+Failure handling+Idempotency+Observability+Progress tracking+Cost awareness
A longer execution window gives a function more time.
Good architecture determines what happens during that time.
Instead of asking whether a workload is "too long for serverless," it is more useful to ask whether the workload can be modeled as a reliable, observable, asynchronous job.
That is where longer-running Lambda workloads become genuinely interesting.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article