aws / aws/containers-roadmap

[ECS] [Feature Request]: Scale-out action resets scale-in cooldown triggering scale-in action

Open
#2,135 0 comments 4 reactions 0 assignees View on GitHub
ECS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

Currently, when an ECS Auto Scaling group scales out, it resets the cooldown of the scale-in process for that group. This behavior can lead to unintended consequences when there are multiple scaling policies based on different metrics. For example, when a scale-out policy with a shorter cooldown than the scale-in cooldown is triggered, it can immediately lead to a scale-in action by another policy, creating a loop of scaling actions.

**Which service(s) is this request for?**
This could be ECS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

A TargetScaling policy based on CPU_AVERAGE that scales both in and out.
A StepScaling policy based on a CloudWatchMetric for GetRecords.IteratorAgeMilliseconds of a Kinesis stream that only scales out.

If the first policy triggers a scale-out action, it resets the scale-in cooldown, and if the second policy meets its criteria for scale-in at that moment, it will immediately trigger a scale-in action. This loop can repeat indefinitely if the scale-out policy has a shorter cooldown than the scale-in cooldown.

**Are you currently working around this issue?**
How are you currently solving this problem?

To mitigate this behaviour, we have increased the scale-out cooldown of the StepScaling policy from 1min to 15mins to decrease the frequency which this occurs - as far as I am aware there is no fix for this issue.

**Additional context**
Anything else we should know?

We request a feature enhancement that allows the scale-out process to reset the cooldown of scale-in for its Auto Scaling group without triggering any scaling policies that meet the criteria for a scale-in action. This enhancement would help prevent unintended scaling loops in scenarios where multiple scaling policies are in use, each based on different metrics.

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ECS Auto Scaling behavior described for TargetScaling and StepScaling policies, focusing on how scale-out and scale-in cooldowns interact. Reproduce the scenario using CPU_AVERAGE, GetRecords.IteratorAgeMilliseconds, and the stated cooldowns; done means the requested cooldown handling prevents the unintended scaling loop without suppressing valid scaling actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.