Event-Driven AI Workflows with Amazon EventBridge and AWS Step Functions
Build an event-driven AI workflow on AWS. Use Amazon EventBridge to react to a new S3 file, AWS Step Functions to call Amazon Bedrock with retries and error handling, and DynamoDB to save the result.
Drop a file in S3 and have a summary appear, with retries, error handling and a clear record of every step, without writing a server.
So far in this series, our AI calls have been triggered by a person or a script running by hand. Real systems rarely work that way. A document arrives, a customer submits a form, a nightly job fires, and something needs to happen next, reliably, even when the model is slow or throttled.
Two AWS services are made for this. Amazon EventBridge notices that something happened and routes it to the right place. AWS Step Functions runs the steps that follow, in order, with retries and error handling built in. In this article we'll wire them together into a small workflow: when a text file lands in S3, summarise it with Amazon Bedrock and save the result.
What you need
- The S3 bucket from the earlier article, or any bucket you can change, in the same Region as your Bedrock work
- A DynamoDB table to hold results, with
doc_key(a string) as its partition key. The table from the earlier DynamoDB article is a good model.
- Access to a Bedrock model, with its ID copied from the model's documentation
- The AWS CLI configured
- About an hour
This workflow creates several resources and makes model calls, so set a budget alert first and read the clean-up step at the end.
How the pieces fit
Here's the flow:
- A text file is uploaded to the
raw/folder of your bucket.
- S3 sends an event to EventBridge.
- An EventBridge rule sees it matches a pattern and starts a Step Functions execution.
- The state machine reads the file, asks Bedrock for a summary, and writes it to DynamoDB.
- If Bedrock is throttled, the workflow retries. If something fails for good, it records the failure instead of silently disappearing.
Each service does one job, and none of them needs a server you maintain.
Why not just use a Lambda function?
You could, and for a single quick step that's often the right choice. Step Functions earns its place when you have several steps, need retries with backoff, want to see exactly where a run stalled, or need to wait for something such as a person's approval. Its visual execution history makes debugging much easier than hunting through logs.
There's also a cost angle. A function that sits waiting while a model writes a long answer is billed for that waiting. Step Functions can call Bedrock directly through an optimized integration, so no function is needed in the middle.
Step 1: Send S3 events to EventBridge
S3 doesn't send events to EventBridge unless you ask. Turn it on for your bucket:
Bash
aws s3api put-bucket-notification-configuration \--bucket my-ai-project-dev \--notification-configuration '{"EventBridgeConfiguration": {}}'
Now S3 publishes events, such as "Object Created," to your account's default event bus.
Note that this command replaces the bucket's existing notification settings, so if you already use S3 notifications for something else, include those in the configuration.
Step 2: Create the state machine
A Step Functions workflow is called a state machine, written in a JSON language called Amazon States Language. Each state is a step. Here's one with three working steps and a failure path:
JSON
{"Comment": "Summarise a new text file and save the result","StartAt": "ReadFile","States": {"ReadFile": {"Type": "Task","Resource": "arn:aws:states:::aws-sdk:s3:getObject","Parameters": {"Bucket.$": "$.detail.bucket.name","Key.$": "$.detail.object.key"},"ResultSelector": { "text.$": "$.Body" },"ResultPath": "$.file","Next": "Summarise"},"Summarise": {"Type": "Task","Resource": "arn:aws:states:::bedrock:invokeModel","Parameters": {"ModelId": "YOUR_MODEL_ID","Body": {"anthropic_version": "bedrock-2023-05-31","max_tokens": 400,"messages": [{"role": "user","content": [{"type": "text","text.$": "States.Format('Summarise this text in three bullet points: {}', $.file.text)"}]}]}},"ResultSelector": { "summary.$": "$.Body.content[0].text" },"ResultPath": "$.result","Retry": [{"ErrorEquals": ["Bedrock.ThrottlingException"],"IntervalSeconds": 5,"MaxAttempts": 4,"BackoffRate": 2}],"Catch": [{ "ErrorEquals": ["States.ALL"], "ResultPath": "$.error", "Next": "Failed" }],"Next": "SaveSummary"},"SaveSummary": {"Type": "Task","Resource": "arn:aws:states:::dynamodb:putItem","Parameters": {"TableName": "summaries","Item": {"doc_key": { "S.$": "$.detail.object.key" },"summary": { "S.$": "$.result.summary" }}},"End": true},"Failed": {"Type": "Fail","Error": "SummaryFailed","Cause": "The model call failed after retries."}}}
In the Step Functions console, create a state machine, choose the code view and paste this in. Walking through it:
- ReadFile fetches the object from S3, using the bucket and key that arrive in the event.
- Summarise calls Bedrock through the optimized integration. If it hits a throttling error, it waits five seconds and tries again, doubling the wait each time, up to four attempts. Anything else that goes wrong falls through to the failure state.
- SaveSummary writes the result to DynamoDB.
- Failed ends the run clearly marked as failed, instead of leaving you guessing.
A few important cautions about this example:
- The
Bodyis specific to the model. Unlike the Converse API we used earlier, InvokeModel needs the request format that model expects. The format shown is for Anthropic models on Bedrock. ReplaceYOUR_MODEL_IDand the body with what your model's documentation specifies.
- Check the model ID format. Some models need an inference profile ID or ARN here. Copy it from the model's documentation.
- Payloads are limited. Step Functions passes data between states in a payload with a size limit of 256 KB, so this design suits small text files. For large documents, split them up or process them differently.
- Step Functions can also call other Bedrock APIs through its AWS SDK integrations. Check the current documentation if you'd prefer a model-neutral call.
Step 3: Give the state machine permission
The state machine runs as an IAM role, and that role needs to be able to do exactly three things:
JSON
{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": "s3:GetObject","Resource": "arn:aws:s3:::my-ai-project-dev/raw/*"},{"Effect": "Allow","Action": "bedrock:InvokeModel","Resource": "arn:aws:bedrock:us-east-1::foundation-model/YOUR_MODEL_ID"},{"Effect": "Allow","Action": "dynamodb:PutItem","Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/summaries"}]}
The role's trust policy must allow
states.amazonaws.com to assume it. If you let the console create the role, it adds that for you, and you can then edit the permissions to match the above. Replace the Region, account number, bucket and model with your own, and if your model is called through an inference profile, the permissions need to cover that too.Step 4: Create the EventBridge rule
Save this event pattern as
pattern.json . It matches new objects under raw/ in your bucket:JSON
{"source": ["aws.s3"],"detail-type": ["Object Created"],"detail": {"bucket": { "name": ["my-ai-project-dev"] },"object": { "key": [{ "prefix": "raw/" }] }}}
Then create the rule and point it at your state machine:
Bash
aws events put-rule --name summarise-new-docs --event-pattern file://pattern.jsonaws events put-targets --rule summarise-new-docs \--targets "Id"="1","Arn"="YOUR_STATE_MACHINE_ARN","RoleArn"="YOUR_EVENTBRIDGE_ROLE_ARN"
The EventBridge role needs one permission:
states:StartExecution on that one state machine. Nothing more.The event pattern is your filter. Keep it narrow, so that only the files you intend start a run. A rule that matches every object in the bucket will happily summarise things you didn't mean it to.
Step 5: Try it
Upload a small text file:
Bash
aws s3 cp ./notes.txt s3://my-ai-project-dev/raw/notes.txt
Then open the Step Functions console, find the latest execution and watch it run. The graph view colours each state as it succeeds or fails, and you can click any step to see exactly what went in and what came out. That's the real payoff of this design. When something goes wrong, you can see which step it was and what data it had.
Check your DynamoDB table for the summary.
Test the unhappy paths too. Upload an empty file, a very large one and a file in the wrong format, and see what happens.
Things to design for
Events can arrive more than once. EventBridge delivers events at least once, so occasionally the same event may trigger two runs. Make your workflow safe to repeat. Here, writing the same key twice just overwrites the same summary, which is fine. For steps that aren't repeat-safe, such as sending an email, add a check, for example a conditional write that only succeeds the first time.
Failures need a home. EventBridge can retry delivery and send events it couldn't deliver to a dead-letter queue. And within the workflow, the failure state marks the run as failed. Decide who gets told, and set up an alarm on failed executions.
Standard or Express? Standard workflows can run for a long time and keep a full history, which suits AI jobs that wait on slow steps. Express workflows are for very high volumes of short runs. For model calls, start with Standard.
Waiting for a person. For actions that need approval, Step Functions supports a callback pattern, where the workflow pauses until something sends back a token. It's the standard way to put a human check into an AI process, such as approving a drafted reply before it's sent.
Many files at once. If hundreds of files arrive together, you'll hit Bedrock throttling. The retry settings help, and it's worth looking at limiting concurrency or using a queue in front of the workflow.
Don't loop forever. If a workflow can send its output back into the bucket that triggers it, you can create an endless cycle. Keep inputs and outputs in different folders, as the
raw/ filter above does.Scheduled runs
Not everything starts from an event. For something that should run every night, such as a daily digest, use EventBridge Scheduler to start the same state machine on a schedule, with no S3 involved.
What it costs
You pay for several things: Step Functions charges per state transition for Standard workflows, Bedrock charges for tokens, and DynamoDB and S3 have their own small charges. EventBridge has its own event pricing, so check the current pricing page. The model call usually dominates. Add all of this to your budget alerts and tag the resources, as in the cost article.
Common mistakes
- A rule that's too broad. Use a precise event pattern.
- Forgetting to enable EventBridge notifications on the bucket. Nothing will fire.
- Wrong body format for the model. InvokeModel needs the model's own format.
- Putting a large file through the payload. Mind the 256 KB limit.
- No retry on throttling. Bedrock throttles under load, so retry with backoff.
- Giving the roles broad permissions. Each role should do one job.
- Output that re-triggers the input. Separate your folders.
Clean up
When you've finished, delete the EventBridge rule, the state machine, the IAM roles you created for it, the DynamoDB table and any test files. Then check your bill after a day or two to make sure nothing is still running.
A checklist
- Is the bucket sending events to EventBridge?
- Is the event pattern narrow enough?
- Does the model call use the right body format and ID?
- Do retries and a failure state exist?
- Does each role have only the permissions it needs?
- Is the workflow safe if the same event arrives twice?
- Do you have an alarm on failed runs, and a budget alert?
Where we go next
In the next article we'll look at how to expose your AI function to the outside world safely, with Amazon API Gateway and AWS Lambda.
EventBridge, Step Functions and Bedrock integrations, limits and pricing change often. Check the current AWS documentation before you follow any step here, and test everything in a non-production account first.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article