Feat: Use built-in priority classes for critical EKS-A components and limit the consumption of these classes for normal workload by default
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 328
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 9
Description
**What would you like to be added**:
By default, Kubernetes ships with two priority classes which is also true for EKS-A:
```
$ kubectl get priorityclasses.scheduling.k8s.io -A
NAME VALUE GLOBAL-DEFAULT AGE
system-cluster-critical 2000000000 false 29d
system-node-critical 2000001000 false 29d
```
These should be used for [critical components/add-ons of the cluster](https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/) to make sure that these components are not evicted first in case of a resource pressure.
Currently, EKS-A does not use these priority classes for all of it's infra components:
```
$ kubectl get pods -n kube-system kube-vip-kls107 -o yaml|grep priority
priority: 0
$ kubectl get pods -n eksa-system eksa-controller-manager-6bb5cb4fb4-lx8q5 -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
$ kubectl get pods -n eksa-system tink-controller-5d876c7b44-z55h8 -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
$ kubectl get pods -n eksa-system kube-vip-pm5cx -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
```
IMO, it would be a stability improvement to make sure that all critical EKS-A components are using one of these priority classes.
On top of this, with the current config, any workload on EKS-A could start to use these built-in priority classes which could result into a situation where EKS-A components are not able to run anymore as K8s scheduler would favor normal workloads in case of a resource pressure. Kubernetes provides a way to prevents this through [AdmissionConfiguration](https://kubernetes.io/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default). It would be advisable to implement this limitation for normal workload for the built-in classes.
**Why is this needed**:
To improve cluster stability of the cluster in case of resource pressure.
Contributor guide
Research direction
No repository files or tests are named. Start by inspecting the Kubernetes resources for kube-vip, eksa-controller-manager, and tink-controller, then compare their priority settings with the linked AdmissionConfiguration guidance. Done means critical EKS-A components use the built-in priority classes and normal workloads cannot consume those classes by default; verify the resulting pod settings with kubectl.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100