AWS Builder Center
Scale Amazon ECS with custom CloudWatch metrics

Scale Amazon ECS with custom CloudWatch metrics

Custom CloudWatch metrics like throttle rates can drive Amazon ECS target tracking policies. Covers the proportionality rule, AWS CDK metric math, and the pitfalls.

SDE at Amazon Ads
You can scale an Amazon Elastic Container Service (Amazon ECS) service on almost any CloudWatch metric — requests per host, throttle rate, queue depth — as long as that metric moves in step with your task count. This article shows how to wire a custom metric into an ECS target tracking policy with the AWS Cloud Development Kit (AWS CDK), and the operational lessons I learned running it in production across multiple services.
AWS Application Auto Scaling ships with predefined target tracking metrics for ECS:  ECSServiceAverageCPUUtilization ,  ECSServiceAverageMemoryUtilization , and  ALBRequestCountPerTarget . Those cover most workloads. But if your real bottleneck is something else — an application-level throughput cap, a downstream throttle, or a connection limit — CPU-based scaling leaves you under-provisioned right when traffic climbs.

When CPU isn't your scaling signal

Predefined metrics fall short in a few specific cases:
  1. Framework-level throughput limits. Your request router or connection pool caps requests per host well below the point where CPU saturates. Scaling on CPU keeps too few tasks running during a spike.
  1. Downstream throttle rate. Your service calls a dependency that rejects requests once per-host load crosses a threshold. If you are being throttled, you want to scale out to spread the load — so the throttle rate itself is a direct scaling signal.
  1. Network or I/O-bound work. The bottleneck is network throughput, disk I/O, or open connections rather than compute.
In each case you need a metric that represents the actual constraint.

The one rule a custom metric must follow

Not every CloudWatch metric works with target tracking. The metric has to move inversely with task count: when the service adds tasks, the metric value must fall, because the load is spread across more hosts. When it removes tasks, the value must rise. AWS documents this as the requirement that the metric be a valid utilization metric proportional to capacity.
Here is why it matters. If your metric does not fall when you scale out, the policy sees the target still exceeded and keeps adding tasks. You get an unbounded scaling loop — burning cost with no improvement in the signal that triggered it.
Metrics that work:
  • TPS per host (total requests per second ÷ task count). Adding tasks lowers per-host TPS. Useful when your service has a known per-host throughput cap and you want each host to stay under it.
  • Throttle rate (throttled requests ÷ total requests). Adding tasks lowers per-host request volume, which lowers the throttle rate.
  • Queue depth per consumer. Adding consumers drains the queue faster.
Metrics that do not work:
  • Total request count (absolute). This is external demand. It stays the same for a given moment no matter how many tasks you run, so the policy can never satisfy it by scaling.
  • P99 latency. May not fall proportionally with more tasks; latency often has floors unrelated to capacity.
  • Error rate from a single downstream. Independent of your task count.

Architecture


The flow is a loop. Your ECS tasks emit a custom metric (TPS/request count, throttle rate) to CloudWatch. A target tracking scaling policy watches that metric — directly or through a metric math expression — against a target value. The policy owns two alarms it creates automatically: a scale-out alarm and a scale-in alarm. When an alarm breaches, the policy triggers Application Auto Scaling, which adjusts the ECS desired count. More tasks lower the per-host metric, and the loop settles at your target.

Two ways to get a custom metric into CloudWatch

There are two ways to produce the metric:
  1. Application-emitted. Your app publishes the metric directly using the CloudWatch API or the embedded metric format (EMF). Simplest when your app already counts requests per host.
  1. CloudWatch metric math. You define the metric as an expression over existing metrics, for example  totalRequests / taskCount . No application change, but both input metrics have to exist.
Target tracking accepts these through the  CustomizedMetricSpecification , which takes either a single metric or a metric math expression.
One limit to know: CloudWatch  SEARCH  expressions are not supported in target tracking policies. Use explicit metric references with known dimensions, or simple math over those metrics.

Prerequisites

  • An AWS account and permissions to manage Application Auto Scaling and ECS.
  • An existing ECS service and a registered scalable target.
  • AWS CDK v2 with TypeScript. Examples below were written against  aws-cdk-lib  2.x on Node.js 20.
  • A custom metric already flowing into CloudWatch (application-emitted or derivable via metric math).

CDK implementation

Example 1: TPS per host with metric math

This policy keeps TPS per host (requests per second per task) at or below 250. The tracked value is total TPS divided across the running tasks.
Watch the units. Target tracking evaluates at 1-minute granularity, so a  Sum  statistic on a request-count metric gives requests per minute. To get true requests per second, divide the per-minute sum by 60 in the expression, as shown below. Keep the metric name, the expression, and the target value on the same unit.
import {
aws_applicationautoscaling as appscaling,
} from 'aws-cdk-lib';

// Assumes `cluster`, `service`, and `this` (a Construct scope) are in scope.
const scalableTarget = new appscaling.ScalableTarget(this, 'ScalableTarget', {
serviceNamespace: appscaling.ServiceNamespace.ECS,
scalableDimension: 'ecs:service:DesiredCount',
resourceId: `service/${cluster.clusterName}/${service.serviceName}`,
minCapacity: 10,
maxCapacity: 500,
});

const tpsPerHostPolicy = new appscaling.CfnScalingPolicy(this, 'TpsPerHostScaling', {
policyName: 'TpsPerHostTargetTracking',
policyType: 'TargetTrackingScaling',
scalingTargetId: scalableTarget.scalableTargetId,
targetTrackingScalingPolicyConfiguration: {
targetValue: 250, // Target: 250 TPS (requests per second) per host.
scaleInCooldown: 300,
scaleOutCooldown: 120, // Must exceed task launch time (image pull + start + health check).
customizedMetricSpecification: {
metrics: [
{
id: 'requestsPerMinute',
metricStat: {
metric: {
metricName: 'TotalRequestCount',
namespace: 'MyService/Traffic',
dimensions: [
{ name: 'ServiceName', value: service.serviceName },
],
},
stat: 'Sum', // Sum over the 1-minute period = requests per minute.
},
returnData: false,
},
{
id: 'taskCount',
metricStat: {
metric: {
metricName: 'RunningTaskCount',
namespace: 'ECS/ContainerInsights',
dimensions: [
{ name: 'ClusterName', value: cluster.clusterName },
{ name: 'ServiceName', value: service.serviceName },
],
},
stat: 'Average',
},
returnData: false,
},
{
id: 'tpsPerHost',
// Divide by 60 to convert requests-per-minute to requests-per-second (TPS).
expression: 'requestsPerMinute / taskCount / 60',
returnData: true, // The value the policy tracks against targetValue.
},
],
},
},
});
Key points:
  • Exactly one metric in the  metrics  array sets  returnData: true . That is the value compared to  targetValue .
  • The inputs ( requestsPerMinute ,  taskCount ) set  returnData: false . They only feed the expression.
  • Application Auto Scaling creates and manages two CloudWatch alarms per policy (one for scale-out, one for scale-in) behind the scenes.
  • I dropped explicit  unit  fields. If you specify a unit that does not match what the emitting application publishes, the metric returns no data and the policy silently stops acting. Only set  unit  when you are certain it matches.

Example 2: throttle rate

If your service calls a dependency that throttles, scale on the throttle rate to keep it under a threshold.
const throttlePolicy = new appscaling.CfnScalingPolicy(this, 'ThrottleRateScaling', {
policyName: 'ThrottleRateTargetTracking',
policyType: 'TargetTrackingScaling',
scalingTargetId: scalableTarget.scalableTargetId,
targetTrackingScalingPolicyConfiguration: {
targetValue: 0.01, // Keep throttle rate at or below 1%.
scaleInCooldown: 300,
scaleOutCooldown: 120,
customizedMetricSpecification: {
metrics: [
{
id: 'throttledRequests',
metricStat: {
metric: {
metricName: 'ThrottledRequestCount',
namespace: 'MyService/Downstream',
dimensions: [
{ name: 'ServiceName', value: service.serviceName },
{ name: 'Dependency', value: 'RateLimitedAPI' },
],
},
stat: 'Sum',
},
returnData: false,
},
{
id: 'totalRequests',
metricStat: {
metric: {
metricName: 'TotalRequestCount',
namespace: 'MyService/Downstream',
dimensions: [
{ name: 'ServiceName', value: service.serviceName },
{ name: 'Dependency', value: 'RateLimitedAPI' },
],
},
stat: 'Sum',
},
returnData: false,
},
{
id: 'throttleRate',
expression: 'throttledRequests / totalRequests',
returnData: true,
},
],
},
},
});

Example 3: a direct application-emitted metric

If your app already emits a per-host metric (via EMF or the CloudWatch API), reference it directly — no metric math needed.
const directMetricPolicy = new appscaling.CfnScalingPolicy(this, 'DirectMetricScaling', {
policyName: 'DirectTpsPerHostTracking',
policyType: 'TargetTrackingScaling',
scalingTargetId: scalableTarget.scalableTargetId,
targetTrackingScalingPolicyConfiguration: {
targetValue: 250,
scaleInCooldown: 300,
scaleOutCooldown: 120,
customizedMetricSpecification: {
metricName: 'TpsPerHost',
namespace: 'MyService/Scaling',
dimensions: [
{ name: 'ClusterName', value: cluster.clusterName },
{ name: 'ServiceName', value: service.serviceName },
],
statistic: 'Average',
unit: 'Count/Second',
},
},
});
Simpler, but your application has to compute and emit the per-host value itself.

Running several policies at once

In production you will likely run more than one policy on the same service — say CPU, TPS per host, and throttle rate. Application Auto Scaling combines them with a max-wins rule (confirm the exact wording in the docs before you rely on it):
  • Scale-out: if the CPU policy asks for 100 tasks and the TPS policy asks for 150, you get 150. The most aggressive policy wins.
  • Scale-in: the service scales in only when every policy agrees capacity can drop. If one policy still needs the current size, the fleet holds. This stops one policy from removing capacity another still needs.
import * as cdk from 'aws-cdk-lib';
import { aws_applicationautoscaling as appscaling } from 'aws-cdk-lib';

// CPU policy using a predefined metric, on the same scalable target.
scalableTarget.scaleToTrackMetric('CpuScaling', {
targetValue: 40,
predefinedMetric: appscaling.PredefinedMetric.ECS_SERVICE_AVERAGE_CPU_UTILIZATION,
scaleInCooldown: cdk.Duration.seconds(300),
scaleOutCooldown: cdk.Duration.seconds(120),
});

// TPS-per-host and throttle-rate policies from the examples above attach to the same target.
One caution: do not run two policies that track the same metric with different target values. They will fight — one scales out while the other scales in.

Absorbing traffic spikes

Target tracking is not instant. The scale-out alarm needs 3 consecutive 1-minute datapoints in ALARM before it fires, so a scale-out takes at least ~3 minutes to trigger. During that window your existing fleet has to absorb the spike. If you cannot tolerate that delay, pre-scale by lowering the target value so you run more headroom.
const trafficSpikeMultiplier = 5;
const frameworkLimit = 500; // Max requests-per-host your framework handles.
const targetPerHost = frameworkLimit / trafficSpikeMultiplier; // = 100
A lower target keeps the fleet pre-scaled to absorb a sudden burst without waiting for the reaction time. The trade-off is that you run more tasks than steady state strictly needs. That is a cost-versus-availability call, and you own it.

Operational lessons

After running custom-metric target tracking in production across multiple ECS services, these are the lessons that cost me the most to learn.

1. Scale-out reaction time

Target tracking's scale-out alarm needs 3 consecutive 1-minute datapoints in ALARM before it acts, so scale-out takes at least ~3 minutes to trigger. Give your fleet enough headroom to absorb that window of elevated load.

2. Scale-in is deliberately slow

The scale-in alarm requires 15 consecutive 1-minute datapoints below threshold, so the fleet holds capacity for at least ~15 minutes after load drops. That is intentional — it prevents flapping.

3. Cooldown asymmetry matters

Set  scaleOutCooldown  longer than the time a new task needs to become healthy and serve traffic, and set  scaleInCooldown  longer still. If the cooldown expires before new tasks are healthy, the policy sees the metric still elevated and scales out again — over-provisioning the fleet.
  •  scaleOutCooldown : exceed total task launch time (image pull + container start + health check + any warm-up). If tasks take 90 seconds to become healthy, set at least 120.
  •  scaleInCooldown : longer than scale-out, to avoid pulling capacity right after a spike subsides.

4. Metric gaps stop scaling

If your app stops emitting the metric — during a deployment, or at startup — the policy has no data and stops acting. To avoid that:
  • Emit the metric even at zero traffic (publish 0, do not skip).
  • Make new tasks emit within their first minute.
  • Consider metric math over infrastructure metrics (like Container Insights) as a fallback source.

5. Pair with scheduled scaling for known peaks

For predictable load — daily peaks, weekly events, known campaigns — combine target tracking with scheduled scaling. In CDK, use  scaleOnSchedule :
import { Schedule, TimeZone } from 'aws-cdk-lib';
import { aws_applicationautoscaling as appscaling } from 'aws-cdk-lib';

interface ScalingSchedule {
scheduleExpression: string; // cron() or at() expression
minCapacity: number;
maxCapacity?: number;
timezone?: string;
}

const scalingSchedules: ScalingSchedule[] = [
{ scheduleExpression: 'cron(0 14 * * ? *)', minCapacity: 200, timezone: 'America/Los_Angeles' },
{ scheduleExpression: 'cron(0 6 * * ? *)', minCapacity: 50, timezone: 'America/Los_Angeles' },
];

scalingSchedules.forEach((schedule) => {
// Derive a stable ID from the expression so adding/removing schedules
// doesn't make CloudFormation delete and recreate unrelated actions.
const stableId = schedule.scheduleExpression
.replace(/[^a-zA-Z0-9]/g, '-')
.replace(/-+/g, '-')
.replace(/^-|-$/g, '');

scalableTarget.scaleOnSchedule(`ScheduledScaling-${stableId}`, {
schedule: appscaling.Schedule.expression(schedule.scheduleExpression),
minCapacity: schedule.minCapacity,
maxCapacity: schedule.maxCapacity,
timeZone: TimeZone.of(schedule.timezone ?? 'UTC'),
});
});
Note the imports:  Schedule  for the schedule expression comes from the applicationautoscaling module ( appscaling.Schedule  above), while  TimeZone  is exported from the core  aws-cdk-lib . Verify both against your installed  aws-cdk-lib  version — import locations move between releases.
Use a stable construct ID derived from the schedule expression, not an array index, so reordering the list does not churn unrelated scheduled actions.
When scheduled scaling and target tracking are both active, max-wins applies — the service runs at whichever capacity is higher.

Verifying your policies

After you deploy, confirm the policies work:
  1. CloudWatch alarms. Target tracking creates two alarms per policy (scale-out and scale-in). Check they transition between OK and ALARM as expected.
  1. Scaling activity. Use  describe-scaling-activities  to see actual scaling events and what triggered them.
  1. Metric versus target. Graph your custom metric against the target value to confirm the policy holds equilibrium.
aws application-autoscaling describe-scaling-activities \
--service-namespace ecs \
--resource-id service/my-cluster/my-service \
--max-results 10
Expected output is a JSON list of  ScalingActivities , each with a  StatusCode  (for example  Successful ) and a  Cause  string describing which alarm fired. An empty list means no scaling has happened yet.

Wrapping up

Custom metrics let you use target tracking for workloads where the predefined metrics miss the real constraint. The essentials:
  1. Your metric must move inversely with task count — more tasks, lower per-task value.
  1. Use  CfnScalingPolicy  with  CustomizedMetricSpecification  in CDK for full control.
  1. Metric math lets you derive the metric without touching application code.
  1. Layer multiple policies (max-wins) for defense against different bottlenecks.
  1. Plan for the scale-out reaction window, and bake spike headroom into your target value.
Once the metric and the policy are set up correctly, Application Auto Scaling handles the rest — it creates and manages the alarms, runs the proportional math each interval, and enforces the cooldowns. Your service then scales on the constraint that actually matters, with nothing extra to maintain.
Have questions, or a custom metric you scale on that I didn't cover? Leave a comment.



Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article