Automate bulk cleanup of AWS Step Functions state machines by using AWS Batch

Summary

• Scale: large numbers of inactive state machines make listing and reviewing very slow.
• Operations: finding and removing them by hand across accounts and Regions takes DevOps time on every cleanup cycle.
• Security: they might reference IAM roles that may no longer be needed, and their definitions can contain hard-coded values. Both complicate access reviews and audits.

• Manual deletion in the AWS Management Console becomes impractical beyond a few dozen state machines.
• Ad hoc scripts typically lack error handling, audit logging, and security controls.
• Third-party tools add configuration, maintenance, and licensing overhead.
• No AWS service provides native bulk cleanup of state machines.

Warning: deleting a state machine can't be undone. Before you run this against any account that matters:

1. Test the full flow in a non-production account against dummy state machines.
2. Always run a dry run first, and read every line of the output (the matched count, each state machine name, and the oldest and newest creation dates).
3. Scope the IAM role to the narrowest ARN pattern that works. Never leave a bare *.
4. Always pass --max-per-run explicitly. Don't rely on the defaults.
5. Verify the results after execution, and keep the logs as your audit trail.
6. Don't put this on a schedule until you have completed several reviewed manual runs.

Prerequisites and limitations

Prerequisites

• An active AWS account. Use a non-production account for your first tests.
• The infrastructure shown in the architecture diagram is already deployed, by using any infrastructure as code (IaC) tool: private subnets with the VPC endpoints shown in the diagram, an Amazon ECR repository, an AWS Batch Fargate compute environment with a job queue and job definition, the IAM roles, and the encrypted CloudWatch Logs log group. The person who deploys this infrastructure is assumed to have all the permissions needed to create these resources.
• AWS Command Line Interface (AWS CLI), configured with credentials that can push images to the ECR repository, submit AWS Batch jobs, read the CloudWatch Logs log group, and list state machines.
• Docker, to build and push the container image.
• (Optional) Python with boto3 on your workstation, to develop and test the cleanup script locally.

Limitations

• Deletion is irreversible. Step Functions has no recycle bin. Export any definition you might need before you delete it.
• Selection is by creation date and name. This approach selects state machines created before a cutoff date, optionally narrowed by a name filter. Selecting by tag is possible but isn't covered here.
• Choose the cutoff time zone deliberately. Compare creation times and the cutoff date in the same time zone, and decide whether a date-only cutoff includes the whole day. Leave a margin around the cutoff.
• One Region per run. Each job targets a single Region and account. Run it once for each.
• Running executions are stopped. The script stops running executions before it deletes a state machine, so a long-running workflow that only looks idle will lose its in-flight work.
• Throttling applies. Step Functions API limits can slow large runs. The script retries with exponential backoff.
• Cleanup runs on demand. This pattern has no scheduler. Add one only after several reviewed manual runs.

Notes

• Container Python version: use the same Python version for the container base image as you use to test locally.
• Wildcard ARNs: in a single-account setup, ARN patterns can use * for the account and Region. In multi-account or regulated environments, pin the account ID and Region.
• Testing: create dummy state machines in a non-production account to test the job. See Test with dummy state machines.

Architecture

Target technology stack

• AWS Batch (on AWS Fargate)
• Amazon Elastic Container Registry (Amazon ECR)
• Amazon Virtual Private Cloud (Amazon VPC) with private subnets
• VPC endpoints
  • Interface endpoints for Amazon ECR, AWS Step Functions, and Amazon CloudWatch Logs
  • Gateway endpoint for Amazon Simple Storage Service (Amazon S3)
• AWS Identity and Access Management (IAM)
• AWS Key Management Service (AWS KMS)
• Amazon CloudWatch Logs
• Python (boto3) and Docker

Target architecture

1. An operator or a CI/CD pipeline builds the container image and pushes it to an Amazon ECR repository.
2. The operator submits the cleanup job to the AWS Batch job queue, passing the parameters for the run: the cutoff date, an optional name filter, the maximum number of deletions per run, and whether to execute or only do a dry run.
3. AWS Batch schedules a Fargate task in a private subnet.
4. The Fargate task pulls the container image through the Amazon ECR VPC endpoints. ECR stores image layers in Amazon S3, so the S3 gateway endpoint lets the private subnet download them.
5. The script lists the state machines through the Step Functions VPC endpoint and filters them by creation date and name. In dry-run mode it deletes nothing and logs each state machine that would be deleted, with its creation date. In execute mode it stops any running executions and deletes the matching state machines.
6. The task writes its execution log through the CloudWatch Logs VPC endpoint to a log group that is encrypted with an AWS KMS key. This log is your audit trail.

Design notes

• Private by design. The task runs in private subnets with no direct route to the internet, so the VPC endpoints are its only path to the AWS APIs. This is true even though the task and the state machines are in the same account, because being in the same account doesn't give the subnet a network path. No internet gateway is required.
• Least-privilege IAM. The job role can list and describe state machines across the account, because the script has to see them to filter them. It can delete state machines and stop executions only on the ARN patterns you allow.
• Dry run first. The script does nothing destructive unless you explicitly pass --execute.
• Encrypted, retained logs. The log group uses a customer managed KMS key and keeps logs for a defined retention period, so you have a record of every run.
• No servers to manage. AWS Batch queues the jobs and Fargate runs them to completion, so there is no instance to patch.

Tools

• AWS Batch  runs the cleanup job on Fargate and queues each submitted run.
• Amazon ECR  stores the private container image.
• AWS Step Functions  is the service whose state machines the job lists, stops, and deletes.
• Amazon CloudWatch Logs  collects the job output, which serves as your audit trail.
• AWS KMS  encrypts the log group.
• AWS IAM  controls what the job is allowed to delete.
• Docker  builds the container image.
• AWS CLI  submits jobs and reads logs.
• Python  and boto3  provide the runtime and SDK for the cleanup script.

Additional details for creating your cleanup job

Write the cleanup script

• A required cutoff date. Choose one time zone and use it both for the comparison and for the timestamps you log, so the dates in your logs match the dates you filtered on. Accept a date-only form (state whether the whole day is included) and a date-and-time form.
• An optional name filter, for example a prefix or substring.
• A maximum number of state machines to process in one run, with a modest default.
• An explicit flag to delete. Without it, the script runs as a dry run.
• The AWS Region. Each run targets one Region.

• Dry run is the default. In a dry run the script calls no API that changes anything. It logs one line for each state machine it would delete, with its name and creation time, followed by the total count and the oldest and newest creation times.
• Select predictably. List all state machines with pagination. Keep those created before the cutoff whose names match the filter. Sort them oldest first, and apply the per-run cap.
• Stop executions, then delete. For each selected state machine, list its RUNNING executions (with pagination), stop each one, and then delete the state machine. Stopping an execution needs the StopExecution permission on execution ARNs, not state machine ARNs. If a stop fails, log the error and still attempt the delete. DeleteStateMachine is asynchronous. The state machine moves to DELETING, and Step Functions terminates any remaining executions on their next state transition.
• Retry throttling only. Retry throttling errors with exponential backoff and jitter, and cap the delay. Fail immediately on any other error, such as AccessDenied.
• Isolate failures. One failed delete must not stop the others. Log each failure, and exit with a non-zero code if any state machine failed.
• Bound the parallelism. Use a small, configurable number of workers. Raise it only if throttling allows.
• Run unattended. No interactive prompts, and unbuffered output to standard output so AWS Batch can send it to CloudWatch Logs.

Create the Dockerfile

• Start from a slim, currently supported Python base image.
• Install dependencies from a requirements file with pinned versions. Never bake credentials into the image.
• Copy in only the cleanup script.
• Run as a non-root user.
• Set the script as the ENTRYPOINT and --help as the default CMD. The image then does nothing destructive by default, and the AWS Batch job command supplies the real arguments.
• Set unbuffered Python output.
• Skip HEALTHCHECK, because this is a run-to-completion job and not a service.

Publish the image to Amazon ECR

Test with dummy state machines

• Create 20 to 200 state machines with a minimal definition (for example, a single Pass state) and a shared name prefix such as test-cleanup-. You need an IAM execution role for them.
• Start an execution of one or two of them that keeps running, for example with a Wait state, to confirm that the job stops running executions.
• Run the dry run with the prefix as the filter, and check that every dummy state machine is listed and nothing else is.
• Run the deletion, and confirm that only the dummy state machines are removed. Repeat with a small per-run cap to confirm that the oldest ones go first.

Run the cleanup

Reminder: Deletion can't be undone. Follow the warning at the start of this article, and never skip the dry run.

Before you start

• You tested the script and the job against dummy state machines in a non-production account.
• Your filter and cutoff date are narrow, and --max-per-run is small, for example 10. Raise it only after you review the results.
• You know the name of the CloudWatch Logs log group that holds the job output.

Submit a dry run

Note: The following commands are examples. Replace the placeholders with your own values. The argument names in the job command (--region, --date, --filter, --max-per-run, and --execute) are also examples. Use the argument names that your script defines.

Check the job status

Review the dry-run output

Run the deletion

Verify the result

Repeat as needed

• More matches than the cap: the script processes the oldest first, so run the same dry run and execute sequence again until no matches remain. Review each pass.
• Other Regions or accounts: repeat the whole sequence for each one. A run targets a single Region and account.

Best practices

• Combine the date and the name filter. A date alone can match far more than you intended, especially around time zone edges.
• Use a naming prefix convention. Have the code that creates state machines add a consistent prefix, such as tmp- or job-. Your filter and your IAM patterns can then stay narrow and safe.
• Scope the job role tightly. Allow DeleteStateMachine only on the state machine ARN patterns you want cleaned up. StopExecution acts on executions, so grant it on execution ARNs that carry the same name prefix, not on state machine ARNs. Never use a bare *. In multi-account or regulated environments, also pin the account ID and Region in the ARNs.

• Export definitions you might need. Deletion can't be undone, and it also removes the state machine's versions and aliases. If a definition has any value, retrieve it with DescribeStateMachine and keep it in version control or Amazon S3 first.
• Check for in-flight work. The job stops running executions before it deletes a state machine. Confirm that the matched state machines are truly abandoned, and not long-running workflows that only look idle.

• Test every script change. The image changes only when you rebuild it. Test each change against dummy state machines in a non-production account before you update the image that production jobs use.
• Reference an exact image version. Point the job definition at a unique version tag instead of latest, so you always know which script version a run used.
• Keep the logs. The CloudWatch Logs output is your only record of what a run deleted. Set the retention period to meet your own audit needs, and don't delete the log group while you still need that record.

Troubleshooting

Related resources

• DeleteStateMachine API reference 
• Creating granular permissions for non-admin users in Step Functions 
• Step Functions service quotas 
• Amazon ECR interface VPC endpoints 
• AWS Batch on AWS Fargate