aws / aws/containers-roadmap

[ECS] "Active" Deployments Preventing new Deployment Creation

Open
#2,836 0 comments 0 reactions 0 assignees View on GitHub
ECS
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

Hi ECS Folks,

My service had an issue last month where we couldn't initiate a new deployment, receiving an error message of `5 active deployments exist, unable to create new deployment.` I could not find any documentation about this error case, so I opened a support case to learn more.

I got these details back:

```
When you call "UpdateService" or trigger a "force-new-deployment", ECS creates a new deployment and begins transitioning tasks from the previous deployment(s) to the new one. Previous deployments remain in the ACTIVE state until all of their tasks have been stopped and drained. Once five deployments are simultaneously in a non-completed state, ECS rejects any additional deployment requests with this error.
```

Our service has tasks that self-configure scale-in protection. From what I understand, we started a deployment, which could not fully complete because old tasks were protected. A new deployment was started, and this left the first deployment "Active" until all tasks in that deployment get terminated. This new deployment scenario happened enough times in a row until we reached 5 "Active" deployments, at which point we could not start a new deployment.

My understanding is that ECS already has a cap on the number of task definitions per service. I'm curious if it's possible to prevent the scenario I hit by having a previous deployment merge with the new deployment. The outcome would be preventing this artificial active deployment cap that doesn't appear to have any external documentation today. Thanks!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ECS UpdateService and force-new-deployment behavior described in the issue, including how prior deployments remain ACTIVE while tasks drain. Determine whether merging or otherwise preventing the five-active-deployment limit is a defined change, and establish whether completion should be a behavior change, documentation of the limit, or both.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.