kubernetes / kubernetes/node-problem-detector
scaling down will send unnecessary alerts
- Dominant language
- Go
- Stars
- 3.5k
- Forks
- 702
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 11
Description
The npd component checks the health status of kube-proxy via the /healthz interface
https://github.com/kubernetes/node-problem-detector/blob/2baac6c06751cc6dccd8b0ad9b6d08b6780f1a13/pkg/healthchecker/types/types.go#L83
When cluster-autoscaler performs elastic scaling down, it will add the ToBeDeletedByClusterAutoscaler taint to nodes. When a node has the ToBeDeletedByClusterAutoscaler taint, the /healthz interface of kube-proxy will consider itself to be in an unhealthy state.
https://github.com/kubernetes/kubernetes/blob/d4672d9a9db5687c4561048c4840ef1f8ccb3c75/pkg/proxy/healthcheck/proxy_health.go#L188
The elastic scaling down of cluster-autoscaler is standard behavior and should not trigger any alarms, as it can cause confusion.
Contributor guide
Research direction
Read pkg/healthchecker/types/types.go at the kube-proxy /healthz check, then compare it with the linked Kubernetes proxy_health.go behavior for ToBeDeletedByClusterAutoscaler. Trace how npd turns that health result into an alert and verify that scaling down a tainted node no longer produces an unnecessary alarm.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100