[ECS] [request]: Tasks can run with awslogs driver when CloudWatch Logs is not available
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
The `awslogs` log driver is great, but when CloudWatch Logs is not available it will fail the task and is unable to create new tasks.
It would be great to have to option to keep the task running as well as start new tasks when CloudWatch logs is not available.
**Which service(s) is this request for?**
Fargate, ECS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
In our case we misconfigured the logs VPC endpoint that then caused all tasks of the whole account to fail. But the same thing would happen if CloudWatch Logs itself would have an outage. This is a problem for availability of services. Logging is quite crucial, but for a lot of services it would be better to have them running without logs in such a case instead of dying.
Right now there is no way to configure this. When we use awslogs as the log driver and CloudWatch logs is down the tasks will all fail. To us this is quite a problem as it would mean that CW logs downtime would cause a complete downtime for our product. In addition it causes a complete cold start of the system (all tasks died) which can cause some problems as well.
**Are you currently working around this issue?**
We cannot find any workaround apart from not using awslogs
Contributor guide
Research direction
Start by reviewing the requested ECS and Fargate behavior for the awslogs driver when CloudWatch Logs is unavailable. Define how tasks should continue running and how new tasks should start without logs, including the configuration and failure behavior needed for completion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100