aws / aws/containers-roadmap

[ECS] [Deployment]: ECS deployment circuit breaker should deal with exceptional exit container

Open
#1,206 50 comments 270 reactions 1 assignee Claimed by @nikitadahiya View on GitHub
Deployments ECS Proposed Work in Progress
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
What do you want us to build?

Use Cloudformation to update ECS background task

**Which service(s) is this request for?**
ECS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

As the demo of [*Deployment circuit breaker*](https://aws.amazon.com/cn/blogs/containers/announcing-amazon-ecs-deployment-circuit-breaker/), the container start failed with error:
`docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"exit\": executable file not found in $PATH": unknown.`
Which would be handled by deployment circuit breaker.

However more common case is container start succeed but exceptional exit.

For example, the Dockerfile:
```
FROM alpine:latest
exit 1
```

The current situation is the container stoped with `Essential container in task exit` and marked as failed, but when the second task start, the `failedTasks` count will be reset to 1, which means the circuit breaker threshold will never triggered.

What we expected is, regard running but abnormal exit as failed, and not reset the failedTasks count, then meet the breaker threshold and roll back.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.