cloudfoundry / cloudfoundry/cloud_controller_ng
Allow failure threshold for application health check
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 207
- Forks
- 373
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 56
Description
Issue
We observed a lot of application crashes due to health check (with http request) timeout, but all other http requests were actually working right before crashes, and also all other metrics were good.
Actually, our health check endpoint is quite fast without any other logic, and also we increased the timeout to 20 seconds but it doesn't help too much.
Expected result
We are not sure why some of health checks fail due to timeout, might be CPU throttling.
But we expect that the runtime gives the application another chance to do another health check.
Current result
Application instance would be restarted after only one single health check failure.
Possible Fix
Adding failure threshold for health check, and after health check fails failureThreshold times in a row, the runtime considers that the overall check has failed and the container is not healthy/live.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name a file, test, or entry point. Begin by locating the application health-check timeout and the logic that restarts an instance after one failure, then inspect related configuration and tests. Done means consecutive failures are counted and an instance is considered unhealthy only after the configured threshold is reached.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100