aws / aws/containers-roadmap

[ECS] [request]: allow StopTask API to safely deregister and drain connections

Open
#576 7 comments 34 reactions 0 assignees View on GitHub
ECS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

**Tell us about your request**
Currently when you call `aws ecs stop-task` my understanding is that ECS tells the Agent to immediately send SIGKILL to the container. Ideally, we would like to mark a task to be killed and for ECS to act similar to when a container instance drains. Those steps are:
1. Spin up new task and add to TG
2. Begin draining on old task (respecting the min % healthy threshold)
3. Issue SIGKILL to containers

Since this functionality is used for instance draining I hope it is something I have overlooked or will be easy to adopt.

**Which service(s) is this request for?**
Fargate, ECS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
We have pretty large services (200+ tasks) that have local cache and many connections to downstream resources. Sometimes an app gets in a bad JVM state where GCs are happening more frequent than usual and we want to safely mark one task for termination while staying above our `deploymentConfiguration` healthy percent. Currently, the only way to do so is either drain the entire container instance or `--force-new-deployment`. A whole new deployment is an "expensive" process and if we issue a `StopTask` we see an increase in 5xx errors because no connections are drained.

**Are you currently working around this issue?**
Right now we are doing `aws ecs update-service --force-new-deployment`but as I said its "expensive" in time and resources.

Please let me know if you have any questions!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the requested StopTask behavior alongside the existing container-instance draining flow described in the issue. Done would mean safely draining connections for an individual task while respecting the deploymentConfiguration minimum healthy threshold before termination.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.