aws / aws/eks-anywhere

Feat: Use built-in priority classes for critical EKS-A components and limit the consumption of these classes for normal workload by default

Open
#8,701 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.1k
Forks
328
Avg merge
1d 4h
Merged PRs (30d)
9

Description

**What would you like to be added**:
By default, Kubernetes ships with two priority classes which is also true for EKS-A:
```
$ kubectl get priorityclasses.scheduling.k8s.io -A
NAME VALUE GLOBAL-DEFAULT AGE
system-cluster-critical 2000000000 false 29d
system-node-critical 2000001000 false 29d
```

These should be used for [critical components/add-ons of the cluster](https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/) to make sure that these components are not evicted first in case of a resource pressure.

Currently, EKS-A does not use these priority classes for all of it's infra components:
```
$ kubectl get pods -n kube-system kube-vip-kls107 -o yaml|grep priority
priority: 0
$ kubectl get pods -n eksa-system eksa-controller-manager-6bb5cb4fb4-lx8q5 -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
$ kubectl get pods -n eksa-system tink-controller-5d876c7b44-z55h8 -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
$ kubectl get pods -n eksa-system kube-vip-pm5cx -o yaml|grep -i priori
preemptionPolicy: PreemptLowerPriority
priority: 0
```

IMO, it would be a stability improvement to make sure that all critical EKS-A components are using one of these priority classes.

On top of this, with the current config, any workload on EKS-A could start to use these built-in priority classes which could result into a situation where EKS-A components are not able to run anymore as K8s scheduler would favor normal workloads in case of a resource pressure. Kubernetes provides a way to prevents this through [AdmissionConfiguration](https://kubernetes.io/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default). It would be advisable to implement this limitation for normal workload for the built-in classes.

**Why is this needed**:
To improve cluster stability of the cluster in case of resource pressure.

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are named. Start by inspecting the Kubernetes resources for kube-vip, eksa-controller-manager, and tink-controller, then compare their priority settings with the linked AdmissionConfiguration guidance. Done means critical EKS-A components use the built-in priority classes and normal workloads cannot consume those classes by default; verify the resulting pod settings with kubectl.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.