Amazon ECS Managed Instances Now Supports AWS Trainium and Inferentia

Amazon ECS Managed Instances now supports Inferentia2, Trainium1, and Trainium2 via the capacity provider, with automatic Neuron device assignment through a single resourceRequirements entry in the task definition. This post walks through the setup — including a discrepancy between the announcement and the Developer Guide — along with use cases, a comparison with EKS and SageMaker, and cost considerations.

haruki

Published Jun 15, 2026

Last modified Jun 15, 2026

Introduction

On June 3, 2026, AWS announced support for AWS Trainium and AWS Inferentia in Amazon ECS Managed Instances. You can now run generative AI training and inference workloads on these purpose-built accelerators directly within ECS Managed Instances' fully managed environment.

Running Trainium or Inferentia workloads on ECS previously required specifying Neuron device paths per instance type in your task definition — a friction point for teams that want container-based AI/ML infrastructure without the overhead of managing Kubernetes. This update removes that requirement.

Amazon ECS Managed Instances has been expanding its accelerator support over time: NVIDIA GPU metrics were added earlier, and Neuron chips (Trainium and Inferentia) are now included.

This post covers what changed, how to choose between Inferentia2, Trainium1, and Trainium2, how to configure your setup, and what to expect on the cost side.

What Changed

For anyone who has worked with Trainium or Inferentia on ECS, the instance selection in the capacity provider and Neuron device configuration in task definitions are where the complexity shows up.

Area	Before	After
Instance selection in capacity provider	Inferentia2 / Trainium1 / Trainium2 were not explicitly surfaced	Selectable via accelerator attributes such as acceleratorNames
Neuron device assignment in task definition	Device paths manually specified per instance type using linuxParameters.devices	Automatic assignment of all devices using a single resourceRequirements entry
Activation methods	—	AWS Console / Amazon ECS MCP Server / IaC tools

The Problem Before

When running Trainium or Inferentia on ECS, the standard approach was to specify Neuron device paths in linuxParameters.devices for each instance type. For inf2.xlarge, that meant one path (/dev/neuron0); for trn1.32xlarge, it meant sixteen paths from /dev/neuron0 through /dev/neuron15. The task definition was tightly coupled to the instance type, so switching instances required updating the task definition as well.

How the Update Solves It

With this update, you can select Inferentia2, Trainium1, or Trainium2 through the capacity provider and add a single entry to resourceRequirements in your task definition. ECS automatically discovers all Neuron devices on the instance and assigns them exclusively to that task — no per-instance-type path configuration needed. Changing instance types no longer requires touching the task definition.

One thing to flag: the naming used in the announcement differs from what the Developer Guide specifies. The details are in the setup section below.

Supported Accelerators

Inferentia2, Trainium1, and Trainium2 all run on the AWS Neuron SDK, but they are optimized for different workloads. Note that first-generation Inferentia (Inf1) is only supported on the EC2 launch type and is not available with ECS Managed Instances.

	Inferentia2 (Inf2)	Trainium1 (Trn1)	Trainium2 (Trn2)
Primary use	Inference (LLMs, image generation, etc.)	Training (pre-training, fine-tuning)	Large-scale model training and inference
Max chips per instance	12	16	16
Accelerator memory (max)	384 GB (HBM)	512 GB (HBM)	1.5 TB (HBM3)
Memory bandwidth (max)	9.8 TB/s	9.8 TB/s	46 TB/s

Inferentia2 (Inference)

Inf2 is built for inference and works well for large generative AI models including LLMs and image generation models. Up to 12 Inferentia2 chips can be connected via NeuronLink to support distributed inference across chips. Compared to first-generation Inferentia (Inf1), throughput is up to 4x higher and latency is up to 10x lower.

Trainium1 / Trainium2 (Training)

Trn1 is designed for pre-training and fine-tuning generative AI models. It supports up to 16 Trainium chips and delivers up to 3 petaFLOPS of compute performance in FP16/BF16.

Trn2 is the next generation, targeting models in the hundreds of billions to trillion parameter range. With up to 16 Trainium2 chips, it delivers up to 20.8 petaFLOPS in FP8, 1.5 TB of HBM3 memory, 46 TB/s of memory bandwidth, and 3x the energy efficiency of Trn1.

Getting Started

Prerequisites

To run Neuron workloads on ECS Managed Instances, you need:

An ECS Managed Instances capacity provider
A Linux container application using a machine learning framework supported by AWS Neuron (PyTorch, TensorFlow, etc.)
A Linux task definition

Managed Instances automatically uses an AMI that includes the Neuron driver, so there is no need to specify an AMI manually as you would with the EC2 launch type.

Step 1: Select Your Instance Type in the Capacity Provider

In the capacity provider's launch template, use instanceRequirements to target Neuron-enabled instances. The following attributes are available:

acceleratorManufacturers: set to amazon-web-services to include all instances with AWS accelerators (Inferentia and Trainium)
acceleratorNames: set to inferentia2, trainium, or trainium2 to target a specific chip
allowedInstanceTypes: use wildcards such as inf* and trn* to match by instance type name

Example using allowedInstanceTypes:

1
2
3
4
5
{
    "instanceRequirements": {
        "allowedInstanceTypes": ["inf*", "trn*"]
    }
}

Step 2: Configure Neuron Device Assignment in the Task Definition

Add a resourceRequirements entry to your container definition:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "containerDefinitions": [
    {
      "name": "neuron-app",
      "image": "<your-image>",
      "resourceRequirements": [
        {
          "type": "NeuronDevice",
          "value": "ALL"
        }
      ]
    }
  ]
}

With type set to NeuronDevice and value set to ALL, Amazon ECS automatically discovers all Neuron devices on the instance and assigns them exclusively to this container. Because the task requires exclusive access to all devices, only one Neuron task runs per instance.

Watch Out: NEURON_CORE vs. NeuronDevice

The announcement describes this as adding NEURON_CORE=all to the resourceRequirements section of the task definition. The ECS Developer Guide, however, specifies type: NeuronDevice with value: ALL.

When writing your task definition, use NeuronDevice / ALL as shown in the example above — not the NEURON_CORE string from the announcement.

Note At most one container definition per task definition can specify NeuronDevice in resourceRequirements. You cannot combine resourceRequirements with type NeuronDevice and linuxParameters.devices for Neuron devices in the same task definition.

Verifying the Configuration

Once a task is running, call the DescribeTasks API operation to confirm device assignment. The response includes a neuronDeviceIds field on each container showing the IDs of the assigned Neuron devices.

1
2
3
aws ecs describe-tasks \
    --cluster <cluster-name> \
    --tasks <task-id>

To check device availability at the container instance level, use the DescribeContainerInstances API operation. The registeredResources and remainingResources fields include NEURON_DEVICES.

1
2
3
aws ecs describe-container-instances \
    --cluster <cluster-name> \
    --container-instances <container-instance-id>

Use Cases

Use Case 1: LLM Inference Endpoint (Inf2)

Deploy LLM or code completion models as containers on an Inf2 capacity provider. With the resourceRequirements configuration in place, each task gets exclusive access to all Neuron devices on its instance.

Combine this with ECS Service Auto Scaling to adjust task count — and therefore instance count — in response to request volume. For workloads with variable traffic such as chatbots or code completion APIs, this makes it straightforward to scale inference capacity without over-provisioning. Instances like inf2.24xlarge or inf2.48xlarge support distributed inference across multiple chips via NeuronLink for larger models.

Use Case 2: Fine-Tuning (Trn1 / Trn2)

Run LLM fine-tuning jobs using your own data as ECS tasks on Trn1 or Trn2 instances. Unlike inference, fine-tuning is a batch workload — the instance only needs to be running while the job is active. Once the task stops, ECS Managed Instances moves the instance to an idle state with no new tasks scheduled, and it is eventually terminated.

This makes it practical to use expensive training instances only for the duration of the job — a natural fit for domain-specific model fine-tuning or periodic retraining pipelines.

Use Case 3: MLOps Pipeline

Build an end-to-end ML pipeline where each stage — preprocessing, training, evaluation, and deployment — runs as a separate ECS task on the appropriate instance type. For example: preprocessing on general-purpose instances, training on Trn1/Trn2, and evaluation and deployment on Inf2, orchestrated with AWS Step Functions.

Since each stage is an independent ECS task, the pipeline integrates naturally with existing CI/CD workflows, and the entire pipeline can be managed as code.

Comparing Your Options

How This Differs from EKS and SageMaker

Amazon EKS and Amazon SageMaker are the other common options for running Trainium and Inferentia workloads.

EKS has supported Inferentia and Trainium for some time, but assigning Neuron devices to Pods requires installing the Neuron device plugin via Helm, along with optional components for scheduling extensions, node problem detection, and monitoring. On top of managing the cluster itself, you take on the lifecycle management of these Neuron-specific components.

With ECS Managed Instances, Neuron device discovery and assignment are handled through the resourceRequirements configuration alone — no additional plugins required. Teams without Kubernetes expertise can run Neuron workloads in a container environment without that operational overhead.

SageMaker abstracts ML tasks into managed training jobs and inference endpoints, and has supported Trn1 and Inf2 for some time. If the primary goal is model training or inference serving, it is a reasonable fit. But if your application already runs as microservices on ECS, keeping AI/ML workloads on the same cluster — rather than splitting them out into SageMaker jobs and endpoints — tends to be the more consistent operational choice.

ECS Managed Instances with Trainium and Inferentia sits between these two: more managed than EKS for Neuron workloads, more flexible than SageMaker's job and endpoint model.

Cost Considerations

Pricing Model

Using Trainium or Inferentia with ECS Managed Instances incurs two charges: the underlying EC2 instance cost and the ECS Managed Instances management fee.

On the EC2 side, all standard purchasing options apply — On-Demand, Reserved Instances (1-year or 3-year), Savings Plans, and Spot Instances. The ECS Managed Instances fee is charged independently of those purchasing options, billed per second with a one-minute minimum. EC2 costs and management costs are optimized separately. For current pricing, see the Amazon ECS Managed Instances pricing page .

Cost Optimization Tips

For batch workloads like fine-tuning on Trn1/Trn2, ECS Managed Instances terminates the underlying instance after the task stops, so you pay for training instances only while the job runs. If the job is idempotent, pairing this with Spot Instances can reduce EC2 costs further.

For always-on workloads like LLM inference endpoints on Inf2, pair ECS Service Auto Scaling with scale-in policies to reduce task count during low-traffic periods. If utilization is predictable, Reserved Instances or Savings Plans can lower the EC2 portion of the bill.

Inferentia2, Trainium1, and Trainium2 each span a range of instance sizes with varying chip counts and memory. Matching the instance size to actual workload requirements — rather than defaulting to the largest available — is the most direct way to keep costs in check.

Summary

This update lets you select Inferentia2, Trainium1, and Trainium2 through the ECS Managed Instances capacity provider and automatically assign all Neuron devices to a task with a single resourceRequirements entry. The per-instance-type device path configuration that was previously required is no longer needed.

One thing to keep in mind: the announcement refers to NEURON_CORE=all, but the correct syntax in your task definition is type: NeuronDevice with value: ALL. Use what the Developer Guide specifies.

A practical starting point is a small Inf2 inference endpoint — work through the full setup, confirm device assignment with the DescribeTasks API operation, then expand to fine-tuning or MLOps pipelines from there.

References

# amazon-ecs # generative-ai # gen-ai # machine-learning

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Amazon ECS Managed Instances Now Supports AWS Trainium and Inferentia

haruki

Introduction

What Changed

The Problem Before

How the Update Solves It

Supported Accelerators

Inferentia2 (Inference)

Trainium1 / Trainium2 (Training)

Getting Started

Prerequisites

Step 1: Select Your Instance Type in the Capacity Provider

Step 2: Configure Neuron Device Assignment in the Task Definition

Watch Out: NEURON_CORE vs. NeuronDevice

Verifying the Configuration

Use Cases

Use Case 1: LLM Inference Endpoint (Inf2)

Use Case 2: Fine-Tuning (Trn1 / Trn2)

Use Case 3: MLOps Pipeline

Comparing Your Options

How This Differs from EKS and SageMaker

Cost Considerations

Pricing Model

Cost Optimization Tips

Summary

References