[ECS] [request]: Avoid stalling ECS task deployment when associated ELB resources are deleted before the ECS service definition update
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
I'd like to request one of the following behaviors instead of silently stalling ECS task deployment when associated ELB resources are deleted before the ECS service definition update:
- ELB API returns an error when requested to remove an ELB listener that is referenced from an ELB target group used by existing ECS service
- ECS task deployment works after unregistering already deleted target group
**Which service(s) is this request for?**
ECS and ELB
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
When deleting an ELB target group/listener/load balancer and then unregistering the target group from an ECS service definition, no errors are returned, but ECS task deployments stall.
Once entered into this situation, new task deployments don't start, and no event logs are recorded.
Task deployments are resumed only after manually stopping all existing tasks, which causes service downtime.
Especially this easily happens if these resources are managed as code, e.g. by Terraform, and removing ELB resources and updating ECS service definition at once.
**Are you currently working around this issue?**
After noticing this problem, I split terraform code changes to update ECS service definition first and remove ELB resources later, but it requires two release cycles to apply one change.
**Additional context**
**Attachments**
Reproduction code of the issue: https://gist.github.com/at-wat/c53300dd6bc210da6c1cf5e2fcb5c2d9
Contributor guide
Research direction
Start with the linked reproduction code and trace the sequence of deleting ELB resources, updating the ECS service definition, and deploying tasks. Compare the two requested behaviors and determine which service owns the change; the work is done when the selected behavior prevents the deployment from stalling and is validated against this sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, terraform
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100