[ECS] "Active" Deployments Preventing new Deployment Creation
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
Hi ECS Folks,
My service had an issue last month where we couldn't initiate a new deployment, receiving an error message of `5 active deployments exist, unable to create new deployment.` I could not find any documentation about this error case, so I opened a support case to learn more.
I got these details back:
```
When you call "UpdateService" or trigger a "force-new-deployment", ECS creates a new deployment and begins transitioning tasks from the previous deployment(s) to the new one. Previous deployments remain in the ACTIVE state until all of their tasks have been stopped and drained. Once five deployments are simultaneously in a non-completed state, ECS rejects any additional deployment requests with this error.
```
Our service has tasks that self-configure scale-in protection. From what I understand, we started a deployment, which could not fully complete because old tasks were protected. A new deployment was started, and this left the first deployment "Active" until all tasks in that deployment get terminated. This new deployment scenario happened enough times in a row until we reached 5 "Active" deployments, at which point we could not start a new deployment.
My understanding is that ECS already has a cap on the number of task definitions per service. I'm curious if it's possible to prevent the scenario I hit by having a previous deployment merge with the new deployment. The outcome would be preventing this artificial active deployment cap that doesn't appear to have any external documentation today. Thanks!
Contributor guide
Research direction
Start by reviewing the ECS UpdateService and force-new-deployment behavior described in the issue, including how prior deployments remain ACTIVE while tasks drain. Determine whether merging or otherwise preventing the five-active-deployment limit is a defined change, and establish whether completion should be a behavior change, documentation of the limit, or both.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100