
Scale Amazon ECS with custom CloudWatch metrics
Custom CloudWatch metrics like throttle rates can drive Amazon ECS target tracking policies. Covers the proportionality rule, AWS CDK metric math, and the pitfalls.
You can scale an Amazon Elastic Container Service (Amazon ECS) service on almost any CloudWatch metric — requests per host, throttle rate, queue depth — as long as that metric moves in step with your task count. This article shows how to wire a custom metric into an ECS target tracking policy with the AWS Cloud Development Kit (AWS CDK), and the operational lessons I learned running it in production across multiple services.
AWS Application Auto Scaling ships with predefined target tracking metrics for ECS:
ECSServiceAverageCPUUtilization , ECSServiceAverageMemoryUtilization , and ALBRequestCountPerTarget . Those cover most workloads. But if your real bottleneck is something else — an application-level throughput cap, a downstream throttle, or a connection limit — CPU-based scaling leaves you under-provisioned right when traffic climbs.When CPU isn't your scaling signal
Predefined metrics fall short in a few specific cases:
- Framework-level throughput limits. Your request router or connection pool caps requests per host well below the point where CPU saturates. Scaling on CPU keeps too few tasks running during a spike.
- Downstream throttle rate. Your service calls a dependency that rejects requests once per-host load crosses a threshold. If you are being throttled, you want to scale out to spread the load — so the throttle rate itself is a direct scaling signal.
- Network or I/O-bound work. The bottleneck is network throughput, disk I/O, or open connections rather than compute.
In each case you need a metric that represents the actual constraint.
The one rule a custom metric must follow
Not every CloudWatch metric works with target tracking. The metric has to move inversely with task count: when the service adds tasks, the metric value must fall, because the load is spread across more hosts. When it removes tasks, the value must rise. AWS documents this as the requirement that the metric be a valid utilization metric proportional to capacity.
Here is why it matters. If your metric does not fall when you scale out, the policy sees the target still exceeded and keeps adding tasks. You get an unbounded scaling loop — burning cost with no improvement in the signal that triggered it.
Metrics that work:
- TPS per host (total requests per second ÷ task count). Adding tasks lowers per-host TPS. Useful when your service has a known per-host throughput cap and you want each host to stay under it.
- Throttle rate (throttled requests ÷ total requests). Adding tasks lowers per-host request volume, which lowers the throttle rate.
- Queue depth per consumer. Adding consumers drains the queue faster.
Metrics that do not work:
- Total request count (absolute). This is external demand. It stays the same for a given moment no matter how many tasks you run, so the policy can never satisfy it by scaling.
- P99 latency. May not fall proportionally with more tasks; latency often has floors unrelated to capacity.
- Error rate from a single downstream. Independent of your task count.
Architecture

The flow is a loop. Your ECS tasks emit a custom metric (TPS/request count, throttle rate) to CloudWatch. A target tracking scaling policy watches that metric — directly or through a metric math expression — against a target value. The policy owns two alarms it creates automatically: a scale-out alarm and a scale-in alarm. When an alarm breaches, the policy triggers Application Auto Scaling, which adjusts the ECS desired count. More tasks lower the per-host metric, and the loop settles at your target.
Two ways to get a custom metric into CloudWatch
There are two ways to produce the metric:
- Application-emitted. Your app publishes the metric directly using the CloudWatch API or the embedded metric format (EMF). Simplest when your app already counts requests per host.
- CloudWatch metric math. You define the metric as an expression over existing metrics, for example
totalRequests / taskCount. No application change, but both input metrics have to exist.
Target tracking accepts these through the
CustomizedMetricSpecification , which takes either a single metric or a metric math expression.One limit to know: CloudWatch
SEARCH expressions are not supported in target tracking policies. Use explicit metric references with known dimensions, or simple math over those metrics.Prerequisites
- An AWS account and permissions to manage Application Auto Scaling and ECS.
- An existing ECS service and a registered scalable target.
- AWS CDK v2 with TypeScript. Examples below were written against
aws-cdk-lib2.x on Node.js 20.
- A custom metric already flowing into CloudWatch (application-emitted or derivable via metric math).
CDK implementation
Example 1: TPS per host with metric math
This policy keeps TPS per host (requests per second per task) at or below 250. The tracked value is total TPS divided across the running tasks.
Watch the units. Target tracking evaluates at 1-minute granularity, so a
Sum statistic on a request-count metric gives requests per minute. To get true requests per second, divide the per-minute sum by 60 in the expression, as shown below. Keep the metric name, the expression, and the target value on the same unit.import { aws_applicationautoscaling as appscaling,} from 'aws-cdk-lib';
// Assumes `cluster`, `service`, and `this` (a Construct scope) are in scope.const scalableTarget = new appscaling.ScalableTarget(this, 'ScalableTarget', { serviceNamespace: appscaling.ServiceNamespace.ECS, scalableDimension: 'ecs:service:DesiredCount', resourceId: `service/${cluster.clusterName}/${service.serviceName}`, minCapacity: 10, maxCapacity: 500,});
const tpsPerHostPolicy = new appscaling.CfnScalingPolicy(this, 'TpsPerHostScaling', { policyName: 'TpsPerHostTargetTracking', policyType: 'TargetTrackingScaling', scalingTargetId: scalableTarget.scalableTargetId, targetTrackingScalingPolicyConfiguration: { targetValue: 250, // Target: 250 TPS (requests per second) per host. scaleInCooldown: 300, scaleOutCooldown: 120, // Must exceed task launch time (image pull + start + health check). customizedMetricSpecification: { metrics: [ { id: 'requestsPerMinute', metricStat: { metric: { metricName: 'TotalRequestCount', namespace: 'MyService/Traffic', dimensions: [ { name: 'ServiceName', value: service.serviceName }, ], }, stat: 'Sum', // Sum over the 1-minute period = requests per minute. }, returnData: false, }, { id: 'taskCount', metricStat: { metric: { metricName: 'RunningTaskCount', namespace: 'ECS/ContainerInsights', dimensions: [ { name: 'ClusterName', value: cluster.clusterName }, { name: 'ServiceName', value: service.serviceName }, ], }, stat: 'Average', }, returnData: false, }, { id: 'tpsPerHost', // Divide by 60 to convert requests-per-minute to requests-per-second (TPS). expression: 'requestsPerMinute / taskCount / 60', returnData: true, // The value the policy tracks against targetValue. }, ], }, },});Key points:
- Exactly one metric in the
metricsarray setsreturnData: true. That is the value compared totargetValue.
- The inputs (
requestsPerMinute,taskCount) setreturnData: false. They only feed the expression.
- Application Auto Scaling creates and manages two CloudWatch alarms per policy (one for scale-out, one for scale-in) behind the scenes.
- I dropped explicit
unitfields. If you specify a unit that does not match what the emitting application publishes, the metric returns no data and the policy silently stops acting. Only setunitwhen you are certain it matches.
Example 2: throttle rate
If your service calls a dependency that throttles, scale on the throttle rate to keep it under a threshold.
const throttlePolicy = new appscaling.CfnScalingPolicy(this, 'ThrottleRateScaling', { policyName: 'ThrottleRateTargetTracking', policyType: 'TargetTrackingScaling', scalingTargetId: scalableTarget.scalableTargetId, targetTrackingScalingPolicyConfiguration: { targetValue: 0.01, // Keep throttle rate at or below 1%. scaleInCooldown: 300, scaleOutCooldown: 120, customizedMetricSpecification: { metrics: [ { id: 'throttledRequests', metricStat: { metric: { metricName: 'ThrottledRequestCount', namespace: 'MyService/Downstream', dimensions: [ { name: 'ServiceName', value: service.serviceName }, { name: 'Dependency', value: 'RateLimitedAPI' }, ], }, stat: 'Sum', }, returnData: false, }, { id: 'totalRequests', metricStat: { metric: { metricName: 'TotalRequestCount', namespace: 'MyService/Downstream', dimensions: [ { name: 'ServiceName', value: service.serviceName }, { name: 'Dependency', value: 'RateLimitedAPI' }, ], }, stat: 'Sum', }, returnData: false, }, { id: 'throttleRate', expression: 'throttledRequests / totalRequests', returnData: true, }, ], }, },});Example 3: a direct application-emitted metric
If your app already emits a per-host metric (via EMF or the CloudWatch API), reference it directly — no metric math needed.
const directMetricPolicy = new appscaling.CfnScalingPolicy(this, 'DirectMetricScaling', { policyName: 'DirectTpsPerHostTracking', policyType: 'TargetTrackingScaling', scalingTargetId: scalableTarget.scalableTargetId, targetTrackingScalingPolicyConfiguration: { targetValue: 250, scaleInCooldown: 300, scaleOutCooldown: 120, customizedMetricSpecification: { metricName: 'TpsPerHost', namespace: 'MyService/Scaling', dimensions: [ { name: 'ClusterName', value: cluster.clusterName }, { name: 'ServiceName', value: service.serviceName }, ], statistic: 'Average', unit: 'Count/Second', }, },});Simpler, but your application has to compute and emit the per-host value itself.
Running several policies at once
In production you will likely run more than one policy on the same service — say CPU, TPS per host, and throttle rate. Application Auto Scaling combines them with a max-wins rule (confirm the exact wording in the docs before you rely on it):
- Scale-out: if the CPU policy asks for 100 tasks and the TPS policy asks for 150, you get 150. The most aggressive policy wins.
- Scale-in: the service scales in only when every policy agrees capacity can drop. If one policy still needs the current size, the fleet holds. This stops one policy from removing capacity another still needs.
import * as cdk from 'aws-cdk-lib';import { aws_applicationautoscaling as appscaling } from 'aws-cdk-lib';
// CPU policy using a predefined metric, on the same scalable target.scalableTarget.scaleToTrackMetric('CpuScaling', { targetValue: 40, predefinedMetric: appscaling.PredefinedMetric.ECS_SERVICE_AVERAGE_CPU_UTILIZATION, scaleInCooldown: cdk.Duration.seconds(300), scaleOutCooldown: cdk.Duration.seconds(120),});
// TPS-per-host and throttle-rate policies from the examples above attach to the same target.One caution: do not run two policies that track the same metric with different target values. They will fight — one scales out while the other scales in.
Absorbing traffic spikes
Target tracking is not instant. The scale-out alarm needs 3 consecutive 1-minute datapoints in ALARM before it fires, so a scale-out takes at least ~3 minutes to trigger. During that window your existing fleet has to absorb the spike. If you cannot tolerate that delay, pre-scale by lowering the target value so you run more headroom.
const trafficSpikeMultiplier = 5;const frameworkLimit = 500; // Max requests-per-host your framework handles.const targetPerHost = frameworkLimit / trafficSpikeMultiplier; // = 100A lower target keeps the fleet pre-scaled to absorb a sudden burst without waiting for the reaction time. The trade-off is that you run more tasks than steady state strictly needs. That is a cost-versus-availability call, and you own it.
Operational lessons
After running custom-metric target tracking in production across multiple ECS services, these are the lessons that cost me the most to learn.
1. Scale-out reaction time
Target tracking's scale-out alarm needs 3 consecutive 1-minute datapoints in ALARM before it acts, so scale-out takes at least ~3 minutes to trigger. Give your fleet enough headroom to absorb that window of elevated load.
2. Scale-in is deliberately slow
The scale-in alarm requires 15 consecutive 1-minute datapoints below threshold, so the fleet holds capacity for at least ~15 minutes after load drops. That is intentional — it prevents flapping.
3. Cooldown asymmetry matters
Set
scaleOutCooldown longer than the time a new task needs to become healthy and serve traffic, and set scaleInCooldown longer still. If the cooldown expires before new tasks are healthy, the policy sees the metric still elevated and scales out again — over-provisioning the fleet.-
scaleOutCooldown: exceed total task launch time (image pull + container start + health check + any warm-up). If tasks take 90 seconds to become healthy, set at least 120.
-
scaleInCooldown: longer than scale-out, to avoid pulling capacity right after a spike subsides.
4. Metric gaps stop scaling
If your app stops emitting the metric — during a deployment, or at startup — the policy has no data and stops acting. To avoid that:
- Emit the metric even at zero traffic (publish 0, do not skip).
- Make new tasks emit within their first minute.
- Consider metric math over infrastructure metrics (like Container Insights) as a fallback source.
5. Pair with scheduled scaling for known peaks
For predictable load — daily peaks, weekly events, known campaigns — combine target tracking with scheduled scaling. In CDK, use
scaleOnSchedule :import { Schedule, TimeZone } from 'aws-cdk-lib';import { aws_applicationautoscaling as appscaling } from 'aws-cdk-lib';
interface ScalingSchedule { scheduleExpression: string; // cron() or at() expression minCapacity: number; maxCapacity?: number; timezone?: string;}
const scalingSchedules: ScalingSchedule[] = [ { scheduleExpression: 'cron(0 14 * * ? *)', minCapacity: 200, timezone: 'America/Los_Angeles' }, { scheduleExpression: 'cron(0 6 * * ? *)', minCapacity: 50, timezone: 'America/Los_Angeles' },];
scalingSchedules.forEach((schedule) => { // Derive a stable ID from the expression so adding/removing schedules // doesn't make CloudFormation delete and recreate unrelated actions. const stableId = schedule.scheduleExpression .replace(/[^a-zA-Z0-9]/g, '-') .replace(/-+/g, '-') .replace(/^-|-$/g, '');
scalableTarget.scaleOnSchedule(`ScheduledScaling-${stableId}`, { schedule: appscaling.Schedule.expression(schedule.scheduleExpression), minCapacity: schedule.minCapacity, maxCapacity: schedule.maxCapacity, timeZone: TimeZone.of(schedule.timezone ?? 'UTC'), });});Note the imports:
Schedule for the schedule expression comes from the applicationautoscaling module ( appscaling.Schedule above), while TimeZone is exported from the core aws-cdk-lib . Verify both against your installed aws-cdk-lib version — import locations move between releases.Use a stable construct ID derived from the schedule expression, not an array index, so reordering the list does not churn unrelated scheduled actions.
When scheduled scaling and target tracking are both active, max-wins applies — the service runs at whichever capacity is higher.
Verifying your policies
After you deploy, confirm the policies work:
- CloudWatch alarms. Target tracking creates two alarms per policy (scale-out and scale-in). Check they transition between OK and ALARM as expected.
- Scaling activity. Use
describe-scaling-activitiesto see actual scaling events and what triggered them.
- Metric versus target. Graph your custom metric against the target value to confirm the policy holds equilibrium.
aws application-autoscaling describe-scaling-activities \ --service-namespace ecs \ --resource-id service/my-cluster/my-service \ --max-results 10Expected output is a JSON list of
ScalingActivities , each with a StatusCode (for example Successful ) and a Cause string describing which alarm fired. An empty list means no scaling has happened yet.Wrapping up
Custom metrics let you use target tracking for workloads where the predefined metrics miss the real constraint. The essentials:
- Your metric must move inversely with task count — more tasks, lower per-task value.
- Use
CfnScalingPolicywithCustomizedMetricSpecificationin CDK for full control.
- Metric math lets you derive the metric without touching application code.
- Layer multiple policies (max-wins) for defense against different bottlenecks.
- Plan for the scale-out reaction window, and bake spike headroom into your target value.
Once the metric and the policy are set up correctly, Application Auto Scaling handles the rest — it creates and manages the alarms, runs the proportional math each interval, and enforces the cooldowns. Your service then scales on the constraint that actually matters, with nothing extra to maintain.
Have questions, or a custom metric you scale on that I didn't cover? Leave a comment.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article