aws / aws/aws-cdk

(aws-ecs): hanging on deleting a stack with ASG capacity provider

Open
#18,179 18 comments 12 reactions 1 assignee Claimed by @ryparker View on GitHub
@aws-cdk/aws-ecs bug effort/large p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### What is the problem?

The deletion of stack with `AsgCapacityProvider` hangs unexpectedly.

It is surprising as we didn't have such an issue with now deprecated `addCapacity` and we have no ECS tasks in ASG when we delete the stack.

The behaviour seems to be caused by the default `enableManagedTerminationProtection = true`.

See the discussion in the original closed issue and my unaddressed comment: https://github.com/aws/aws-cdk/issues/14732#issuecomment-991402770.

### Reproduction Steps

Please see https://github.com/aws/aws-cdk/issues/14732.

In short, try to delete the stack with ECS cluster which uses `AsgCapacityProvider` defaults.

### What did you expect to happen?

Either:

* CloudFormation does not hang but fails as fast as possible with an error message about the termination protection.
* The stack is successfully deleted as there are no running ECS tasks anymore.

### What actually happened?

The CF stack got stuck in DELETE_IN_PROGRESS.

### CDK CLI Version

2.3.0

### Framework Version

2.3.0

### Node.js Version

v16.8.0

### OS

macOS

### Language

Java

### Language Version

11.0.8

### Other information

### Workaround

My current workaround: set `AsgCapacityProvider` `enableManagedTerminationProtection = false`.

### Documentation questions/enhancement requests

From https://docs.aws.amazon.com/cdk/api/latest/docs/aws-ecs-readme.html (emphasis mine):

> By default, an Auto Scaling Group Capacity Provider will manage the Auto Scaling Group's size for you. **It will also enable managed termination protection, in order to prevent EC2 Auto Scaling from terminating EC2 instances that have tasks running on them.** If you want to disable this behavior, set both enableManagedScaling to and enableManagedTerminationProtection to false.

1) It's not fully clear from the description that the flag simply disables deletion of ASG. I got an incorrect impression that it somehow cleverly understands that there are no ECS tasks running and allows deletion in such case.
1) What are the risks of turning this protection off? E.g. we don't want ECS tasks to shut down at random times.
1) Is it OK to set `enableManagedTerminationProtection=false` + `enableManagedScaling=true`? It seems to work but is against the documentation ("If you want to disable this behavior, set both enableManagedScaling to and enableManagedTerminationProtection to false.").

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.