[ECS] [request]: allow StopTask API to safely deregister and drain connections
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
**Tell us about your request**
Currently when you call `aws ecs stop-task` my understanding is that ECS tells the Agent to immediately send SIGKILL to the container. Ideally, we would like to mark a task to be killed and for ECS to act similar to when a container instance drains. Those steps are:
1. Spin up new task and add to TG
2. Begin draining on old task (respecting the min % healthy threshold)
3. Issue SIGKILL to containers
Since this functionality is used for instance draining I hope it is something I have overlooked or will be easy to adopt.
**Which service(s) is this request for?**
Fargate, ECS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
We have pretty large services (200+ tasks) that have local cache and many connections to downstream resources. Sometimes an app gets in a bad JVM state where GCs are happening more frequent than usual and we want to safely mark one task for termination while staying above our `deploymentConfiguration` healthy percent. Currently, the only way to do so is either drain the entire container instance or `--force-new-deployment`. A whole new deployment is an "expensive" process and if we issue a `StopTask` we see an increase in 5xx errors because no connections are drained.
**Are you currently working around this issue?**
Right now we are doing `aws ecs update-service --force-new-deployment`but as I said its "expensive" in time and resources.
Please let me know if you have any questions!
Contributor guide
Research direction
Start by reviewing the requested StopTask behavior alongside the existing container-instance draining flow described in the issue. Done would mean safely draining connections for an individual task while respecting the deploymentConfiguration minimum healthy threshold before termination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100