
Graceful scale-in for voice AI agents on Amazon ECS with task scale-in protection
A voice AI call can hold an Amazon ECS task for 10 minutes, and new calls keep arriving every few seconds. This post shows how to combine task scale-in protection with application-controlled draining, so a task stops taking calls before Amazon ECS stops the task.
By Mahima Chaturvedi and Sahil Wasan
Introduction
A voice AI agent keeps a live media stream open for the length of a call, running speech-to-text, a large language model, and text-to-speech against it in a loop. That stream belongs to the task that placed or answered the call. It can't move to another task, so when the task stops, the call drops.
Scaling out is straightforward for this workload; scaling in is not. When you lower the desired count of an Amazon Elastic Container Service (Amazon ECS) service, Amazon ECS chooses which task to stop, without knowing which tasks are mid-call. Amazon ECS task scale-in protection lets a task opt out, and the common pattern is to protect a task while it has work and release protection when it goes idle. That pattern assumes tasks go idle. A service where new calls arrive every few seconds, each lasting up to 10 minutes, rarely has an idle task.
In this post, we show how the application can choose the task to remove, drain it, and only then hand it to Amazon ECS. Amazon ECS still starts and stops every task on AWS Fargate. Task scale-in protection makes sure the task it stops is the one the application chose.
Why the usual approaches fall short
CPU and memory don't measure call capacity. A task holding many open calls, most of them silence, can look underused while every call slot is taken. A policy on either metric can scale in during live calls and miss the moment the fleet is full. Concurrent calls per task is the signal that tracks capacity.
Protecting a task whenever it has calls stalls scale-in. Under continuous arrivals, finished calls are replaced within seconds, so a task's active call count rarely reaches zero. Every task stays protected, and the service events report that Amazon ECS is unable to scale in because tasks are under protection.

Figure 1: Why protection tied to "any active call" stalls scale-in under continuous traffic
The default deregistration delay is shorter than a call. When Amazon ECS stops a task behind an Application Load Balancer, it first deregisters the task. The load balancer stops sending it new connections, and existing connections can continue for the deregistration delay, 300 seconds by default. A 10-minute call outlives that, and a call still connected when the delay ends drops, because Amazon ECS then stops the task.
A long deregistration delay alone isn't enough. Raising the delay above the longest call protects calls that are already connected, but it doesn't stop the departing task from dialing. Nothing gives the chosen task a guaranteed head start before Amazon ECS deregisters it, and any call that connects after deregistration fails: the load balancer no longer routes new connections to that task, and no other task can serve the call. The dialer has to stop, and its ringing calls have to connect, before deregistration. That ordering is only possible if the application knows in advance which task will go.
Solution overview
The application already knows its backlog and which tasks carry which calls, so it owns two decisions: how many tasks to run, and which task to remove.
UpdateService can lower the count but can't name a task. Task scale-in protection closes that gap. Every task protects itself as soon as it starts. To remove one, a capacity controller drains it, releases protection on that task alone, and then lowers the desired count, which leaves Amazon ECS exactly one task it can stop.
Figure 2: Solution architecture
Component | Responsibility |
|---|---|
Capacity controller | Sets the task count, and chooses and drains the task to remove |
Shared state store (Valkey) | Records each task's state: ACCEPTING, DRAINING, UNPROTECTED, COUNT_LOWERED |
Dispatcher | Assigns queued call requests to ACCEPTING tasks only |
Calling task | Protects itself, renews protection, and stops its own dialer when draining |
Amazon ECS | Starts and stops tasks, and never stops a protected task during scale-in |
Application Load Balancer | Routes new connections, and holds existing ones open for the deregistration delay |
The drain window comes from the target group: Amazon ECS waits out the deregistration delay before it stops a registered task. A service without a load balancer gets only the
stopTimeout , at most 120 seconds on Fargate, so this approach needs the calling tasks registered with a target group.Prerequisites
- An Amazon ECS service on AWS Fargate in private subnets with a route to the Amazon ECS API (a NAT gateway or Amazon ECS interface VPC endpoints), behind an Application Load Balancer with an HTTPS listener. Carrier media streams, such as Twilio Media Streams, connect over secure WebSocket (
wss://), which Application Load Balancers support natively.
- A way for your carrier to send a placed call's media stream to the task that placed it. A load balancer listener rule can't route to a single task, and the tasks run in private subnets, so this typically means a small proxy that looks up the call ID and forwards the stream to the placing task. With Twilio, carry the call ID in the stream URL's path: Twilio stream URLs don't accept a query string, and a
<Parameter>value arrives only in the stream's start message, after the connection opens. The sample assumes this routing but doesn't build it.
- No Service Auto Scaling policies on the service, because the controller is the only writer of the desired count. Turn off Availability Zone rebalancing as well, by setting
availabilityZoneRebalancingtoDISABLEDon both new and existing services. Amazon ECS turns it on by default when a service is created without the setting, and since September 5, 2025, it has also turned it on for existing eligible services. A rebalance starts a task in the zone with the fewest tasks before it stops one elsewhere. That extra task protects itself, which leaves more protected tasks than the desired count, and the rebalance can't stop any task because every task is protected.
- A call duration cap enforced by the application. This post uses 600 seconds.
- A shared state store that the controller, the dispatcher, and every task can reach. This post uses Valkey on Amazon ElastiCache.
- A task role with
ecs:GetTaskProtectionandecs:UpdateTaskProtection. The controller needs the same two permissions, plusecs:UpdateService,ecs:DescribeServices,ecs:ListTasks, andecs:DescribeTasks.
Scale out: protect first, then accept calls

Figure 3: Scale-out sequence
When the backlog needs more capacity, the controller raises the desired count by one. The new task's first action is to protect itself through the endpoint that the Amazon ECS container agent exposes inside the container:
curl --request PUT --header 'Content-Type: application/json' \ ${ECS_AGENT_URI}/task-protection/v1/state \ --data '{"ProtectionEnabled":true,"ExpiresInMinutes":15}'Amazon ECS recommends this endpoint for tasks that set their own protection, because the agent has built-in retry mechanisms. The endpoint can still return
ThrottlingException or RequestCanceled , so the task retries its first call with backoff. When the task has confirmed its protection and answers its first load balancer health check, it records itself as ACCEPTING, and from then on the dispatcher and the load balancer can send it calls. With our sample image, a new task reached ACCEPTING 30–40 seconds after the controller raised the count.The task renews protection every 5 minutes with a 15-minute expiry, so it stays protected through two failed renewals in a row. If the task hangs, protection lapses within 15 minutes instead of blocking scale-in and deployments for hours.
Answer health checks before connecting to slower dependencies, and set the health check grace period to cover real startup time. In our tests, one task took 36 seconds to make its first connection to the state store and failed its first health checks, so Amazon ECS started a replacement. When the original recovered, the service had four protected tasks for a desired count of three.
That state is worth avoiding. While a service has more protected tasks than its desired count, Amazon ECS can return
DEPLOYMENT_BLOCKED when you update task protection. In our test, requests to turn protection on were refused, and requests to turn it off still worked. Every renewal on all four tasks was refused for 14 minutes, until the newest task's protection expired and Amazon ECS stopped it. The other three renewed less than a second after their own protection had expired. Under live traffic, the task that went would have been whichever one's protection expired first, calls or not. Alarm on a running count above the desired count, and resolve it by releasing a task you choose.Scale in: drain first, then lower the count

Figure 4: Scale-in sequence
Suppose four tasks are running and the controller decides three are enough. It removes one task in eight steps:
- Nominate. The controller picks the least-busy ACCEPTING task, D, and records it as DRAINING before doing anything else, so a restarted controller can resume the sequence.
- Stop admission. The dispatcher stops assigning call requests to D. D sees its DRAINING state, stops its dialer, and stops renewing its protection. It keeps passing its health check, so Amazon ECS doesn't replace it while its existing calls finish.
- Let placed calls connect. A call that D has already dialed may still be ringing, and only D can serve its media stream. The controller waits until none of D's placed calls are still connecting, and until any protection renewal that D sent before it saw DRAINING has finished. A placed call counts as connecting until its media stream arrives or its ring time plus a short grace period ends.
- Release. The controller records the target count of 3, calls
UpdateTaskProtectionwithprotectionEnabled=falsefor D, checks the response for failures, and records UNPROTECTED.
- Lower the count. The controller confirms that D is unprotected and that A, B, and C are protected, releasing D again if a late renewal protected it. If the desired count is still 4, it sets the desired count to the recorded target with
UpdateService, and records COUNT_LOWERED. D is the only task Amazon ECS can stop.
- Deregister. Amazon ECS deregisters D from the target group, which took 13 to 62 seconds after the count dropped in our tests. Until then, the load balancer can still route new connections to D, and D serves them, because each one ends at the 600-second cap, before the 900-second delay does. After that, it sends none, and D's existing calls end naturally or at the cap.
- Stop. Amazon ECS waits for the target to leave the draining state. Elastic Load Balancing can finish deregistering a target with no active connections early, but the target stays in draining until the delay expires, and in our tests Amazon ECS waited out the full deregistration delay (180 seconds) even when D's calls ended sooner. Plan for the full delay. Amazon ECS then sends SIGTERM, and D flushes its state and exits within 120 seconds.
- Clean up. When
DescribeTasksreports D'slastStatusas STOPPED, the controller removes it from the registry and starts a cooldown before the next removal.
The heart of steps 4 and 5 is a few API calls:
ecs = boto3.client("ecs")
# Step 4: release protection on the drain candidate onlyresp = ecs.update_task_protection(cluster=CLUSTER, tasks=[task_d], protectionEnabled=False)if resp.get("failures"): raise RuntimeError("could not release protection; do not lower the count")
# Before step 5: D must be unprotected and every other task protected (task IDs, not ARNs)prot = {t["taskArn"].rsplit("/", 1)[-1]: t["protectionEnabled"] for t in ecs.get_task_protection(cluster=CLUSTER, tasks=other_tasks + [task_d])["protectedTasks"]}if prot.get(task_d) is not False or not all(prot.get(t) is True for t in other_tasks): raise RuntimeError("protection is not as expected; do not lower the count")
# Step 5: an absolute target recorded before step 4, applied only if the count is unchanged,# so a resumed controller can't lower the count twicecurrent = ecs.describe_services(cluster=CLUSTER, services=[SERVICE])["services"][0]["desiredCount"]if current == desired_before: ecs.update_service(cluster=CLUSTER, service=SERVICE, desiredCount=target_count)elif current != target_count: # equal to the target: an earlier run already lowered it raise RuntimeError("the count changed since this removal started; change nothing")The order carries the design. Admission stops before protection is released, so D can't take on new work once it becomes eligible. The count drops only after D is the sole unprotected task, so Amazon ECS can't stop a busy one. Releasing D first also keeps the number of protected tasks at the new desired count, so the remaining tasks' renewals aren't refused; in our test, every one of them succeeded while D drained. A connection that reaches D before deregistration still finishes in time, because its 600-second cap ends before the 900-second delay does.
Settings that must line up
Setting | Value in this post | Rule |
|---|---|---|
Deregistration delay ( deregistration_delay.timeout_seconds ) | 900 s | Longer than the longest call. The range is 0–3,600 s, and the default is 300 s. |
Application call cap | 600 s | Shorter than the deregistration delay. |
stopTimeout | 120 s | Cleanup time after SIGTERM, not drain time. Fargate allows at most 120 s and defaults to 30 s. |
Protection expiry ( ExpiresInMinutes ) | 15 min | Several renewal intervals long. The range is 1–2,880 min, and the default is 120 min. |
Protection renewal | Every 5 min | Well inside the expiry. |
Placed call window | Ring time plus 10 s | Long enough for the carrier to connect a dialed call's media stream. Step 3 waits for it. |
Availability Zone rebalancing | DISABLED , set explicitly | A rebalance starts an extra task before it stops one, and it can't stop a protected task. Amazon ECS turns it on for new services and eligible existing services unless you set it. |
The load balancer health check should report whether the task's process is healthy, not whether its call slots are full. Amazon ECS replaces a task that fails its load balancer health check, and in our test, task scale-in protection didn't prevent that: a protected task that kept failing its health checks was stopped with the reason "Task failed ELB health checks", 37 seconds after Amazon ECS started its replacement. Keep admission control in the shared store, where the dispatcher reads it.
Also set the application server's keep-alive timeout longer than the load balancer's idle timeout (60 seconds by default). Otherwise, the load balancer can return HTTP 502 errors that look like a scaling problem but aren't.
Operating the controller
The controller now owns the desired count, so its failure modes become yours to manage:
- Run one active controller. Use a lease, and remove one task at a time. Keep the lease short, and have the controller release it on SIGTERM. Set
initProcessEnabledon its container, which runs an init process that forwards signals to it. Otherwise, a stopped controller blocks scaling until its lease expires.
- Keep the count out of your templates. If AWS CloudFormation manages the service, leave
DesiredCountout of the template. Otherwise, every stack update that touches the service overwrites the controller's count. A new service then starts with a desired count of 1, so the controller sets the starting count.
- Persist each step before acting on it. A restarted controller continues from the candidate's last recorded state. Because step 5 sets an absolute target recorded in advance, and applies it only if the count hasn't changed since, a restart can't lower the count twice. Each task records its own status in a separate field, so a stopping task can't overwrite the removal state.
- Back a removal out to add capacity quickly. Until a removal has lowered the count, raising the count would make its recorded target stale, so the controller refuses to scale out. To add capacity in that window, back the removal out: re-protect D and return it to ACCEPTING, which restores capacity at once. After step 5, scale out normally once Amazon ECS has started stopping D.
- Check before lowering the count. A task whose renewals have failed for 15 minutes is unprotected too, and a renewal that was in flight when D saw DRAINING could re-protect D. Waiting for D's last renewal to finish (step 3) and the
GetTaskProtectioncheck catch both, and the controller keeps checking D while it drains, releasing it again if needed.
- Don't scale in while a task is starting. A new task is unprotected until its first protection call, so wait until it reaches ACCEPTING.
- Plan for a slow scale-in. Each removal lasts about one deregistration delay, so with a 900-second delay, the fleet sheds roughly one task every 15 minutes plus the cooldown. Set the delay just above the longest call, not higher.
- Reconcile on a schedule. Compare the registry with
ListTasksand with each task's protection, so unexpected stops and unfinished drains get cleaned up. Treat a task as gone only whenDescribeTasksreports it stopped.
What task scale-in protection doesn't cover
Task scale-in protection applies to scale-in from Service Auto Scaling and to deployments, and in our tests it also held for desired count changes made with
UpdateService . Plan separately for these cases:- Unplanned stops. A crash, an out-of-memory kill, or a host failure stops the task immediately, and its calls drop. Recovering a live call needs call state stored outside the task and depends on your telephony carrier.
- Fargate Spot. A Spot interruption gives a two-minute warning, which is shorter than a call, so run calling tasks on regular Fargate capacity.
- Fargate task retirement. When AWS retires a platform version revision, the tasks running on it are stopped, and you can't opt out. Notifications arrive through AWS Health 7 days ahead by default, which you can extend to 14 days or replace with Amazon EC2 event windows. Event windows are best effort and apply only to retirements scheduled after you set them up. Use the notice window to scale out by one, then remove an affected task with the same sequence.
- Deployments. In a rolling update, protected old tasks aren't stopped until their protection clears or expires, so set
maximumPercenthigh enough for new tasks to start alongside them. In our test, the new tasks protected themselves normally, but every renewal on the old tasks was refused withDEPLOYMENT_BLOCKED. Their protection therefore lapses within its expiry, after which Amazon ECS stops them even if they are still dialing. Release each old task first with steps 1 to 4, without lowering the count. When we did, the deployment completed one deregistration delay later. Blue/green deployments also remove old tasks only once their protection clears, and an AWS CloudFormation stack update that waits on protected tasks for more than 3 hours fails and rolls back.
Testing the sequence
We ran the controller and calling tasks from the accompanying repository on AWS Fargate (ARM64, 0.25 vCPU) in private subnets, behind an HTTPS Application Load Balancer in us-east-1. Three tasks held
wss:// calls while a probe opened a short call every 2 seconds. To keep runs short, we used a 90-second call cap, a 180-second deregistration delay, and protection renewal every 60 seconds. We then asked the controller to remove one task.Check | Result |
|---|---|
Task Amazon ECS stopped | The nominated task only. The other two kept running. |
Calls on the drained task | Both ran to the 90-second cap and closed normally, after the task had been deregistered |
New calls reaching the drained task | 7 while it drained, before deregistration, all served; then 0 of 200 |
Calls the drained task had dialed | 10 connected their media streams after draining began, and none failed |
Count lowered to deregistration | 16.5 s |
Deregistration to SIGTERM | 219 s |
Renewals on the remaining tasks during the drain | 11 succeeded, 0 refused |
Conditions: one run with three tasks, nine held calls, and 240 probe calls, all without errors; a simulated dialer with a 20-second ring time; no queue or dispatcher. Three runs of earlier versions of the code gave the same outcome, with 13, 16.5, and 62 seconds from count lowered to deregistration. In the same run, we also stopped the controller after it lowered the count and scaled out while D was still draining. The new task reached ACCEPTING 35 seconds later, and Amazon ECS still stopped only D.
Before production, compare completed calls against your telephony carrier's records for a drain window. Every call should end naturally or at the application cap.
Conclusion
A voice AI workload can't let Amazon ECS stop a busy task, because its calls can't move, and a long deregistration delay can't stop a departing task from dialing. Protecting every task and releasing only the one the application has drained resolves both: admission stops first, protection is released second, the count drops last, and the deregistration delay lets the remaining calls finish. Keep protected tasks at or below the desired count, keep health checks about process health rather than capacity, and plan separately for crashes, Fargate Spot, and Fargate task retirement.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article