[ECS] [request]: Prioritize Daemon task scheduling above Replica tasks
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
**Tell us about your request**
When our Replica ECS Services scale up and we launch new ECS Container Instances (from an ASG, scaling on *Reservation metrics), sometimes the Replica tasks are launched on the instance fast enough that our Daemon Services do not have a chance to launch on these new hosts. If these replicas use enough CPU/memory, there may not be room for the Daemon services to run.
For example, we have a few daemon services to collect host-level metrics and forward log files. We want these to run on every host. Periodically we see that hosts have been saturated with Replica tasks and there's no room for the Daemon tasks. This means we lack monitoring and visibility into these hosts.
**Which service(s) is this request for?**
This could be ECS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
Every host should have Daemon tasks provisioned on it, reliably.
**Are you currently working around this issue?**
We manually go into the ECS console, review all the (possibly hundreds) of hosts to identify which one is missing Daemon services, then run "Stop Task" for one of the replica tasks on that host in order for Daemons tasks to have room to launch.
Contributor guide
Assessment
This issue has not been assessed yet.