[eks] [issue]: Unstable TopologyAwareHints
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
**Tell us about your request**
I have noticed some weird issues with TopologyAwareHints on EKS. I started with creating a ticket on the kubernetes github as i thought it is not an EKS issue. Later I have tested it with kind but there it works fine as far as I can see.
https://github.com/kubernetes/kubernetes/issues/114215
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
TopologyAwareHints are removed and never restored when EKS clusters are scaled. This can be either a CAS scaling or a spot termination.
In Short:
Hints are present on endpointslices then a spot termination comes around and the hints are removed(this could be valid due to the saveguards of hints) tho the hints are never restored. EKS controller logs show nothing.
Then when I recreate(delete) one of the pods of where the hints should be there the hints are restored.
I would think the hints needs to be restored when the cluster is restored to its correct balance without manual intervention.
See original ticket for more information.
**Are you currently working around this issue?**
No WO
**Additional context**
deployment has 6 replica's 2 in each availability zone
All nodes are tagged correctly with the topology.kubernetes.io/zone label
The cluster has a total of 9 nodes, 6 for workload and 3 dedicated for ingress-nginx(so non of the workload pods can be scheduled here(taints))
Cluster is deployed in a total of 3 zones 1a, 1b, 1c
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
Contributor guide
Research direction
Start by reading the linked Kubernetes issue and reviewing the reported EKS controller behavior around cluster scaling and spot termination. Reproduce the six-replica, three-zone setup if access is available; done means TopologyAwareHints are restored automatically after the cluster returns to balance, without deleting a pod.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100