Properly Handle Pods with schedulingGates
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 1.2k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 2
Description
/area autoscale
## Describe the feature
I would like to request to appropriately handle the Pods with [shcedulingGates](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-scheduling-readiness/) (a.k.a gated pods).
I'm wondering if the KPA should not scale deployment with gated pods into zero.
Now, as a workaround, we can avoid the accident automatically scale-in by setting the maximum value (2147483647) in the `progressDeadlineSeconds`.
## background
Currently, the Knative Pod Autoscaler (KPA) automatically scales the not ready pods due to unschedulable into zero based on the `progressDeadlineSeconds` to improve better resource consumption, and the KPA doesn't consider the reason why the Pods are not schedulable.
Additionally, we can not specify null in the `progressDeadlineSeconds` because the field is propagated to Deployment as well.
https://github.com/knative/serving/blob/3aa9210a0d1368c01ec6a7ccfa9e3db5a564a5a5/pkg/reconciler/autoscaling/kpa/scaler.go#L199-L204
https://github.com/knative/serving/blob/3aa9210a0d1368c01ec6a7ccfa9e3db5a564a5a5/pkg/deployment/config.go#L102
However, the unschedulable phase due to the scheduling gate is intentional behavior because the external controller, like the quota management/queuing system, stops scheduling the Pods into any Nodes.
Based on the current KPA implementation, once the gated pod is removed by scaled deployment into zero, the quota management/queueing system could never pop the Pods into the cluster.
Contributor guide
Research direction
Start by reading the KPA scaling logic in pkg/reconciler/autoscaling/kpa/scaler.go around the linked lines, then review progressDeadlineSeconds handling in pkg/reconciler/autoscaling/config.go. Trace how gated Pods are treated when deployments scale toward zero, and define tests that demonstrate the intended behavior for Pods with schedulingGates without losing the existing unschedulable-pod handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100