[EKS] [request]: use resource capacity not ec2 family instance families
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
Creation of a new ec2 instance family for EKS that would not be bound to tradicional families with pre-set sizes.
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
Fargate already lets practitioners chose VMs based on the Pods resource needs.
But for EKS, the underlying instances are still based on tradicional EC2 instance families.
With the extra abstraction of using things like [Karpenter](https://karpenter.sh) , practitioners and developer teams only care about their Pods resources, and let karpenter binpack and chose the cheapest option available between spot and on-demand.
So if AWS was to provide fully dynamic instances that would fit just right the resources that kube-schedule is looking to schedule for new pods, us clients would be paying just for the memory/cpu requested, just like [fargate](https://aws.amazon.com/fargate/pricing/#:~:text=regional%2Dproduct%2Dservices/-,Pricing%20Details,-Pricing%20is%20based).
Since Karpenter does consolidation for us, if Pods were decommissioned, those instances would be replaced with newer ones perfect size fit for the remaining Pods left running.
Say for example you schedule 4 replicas requesting 8GiBs of RAM.
Karpenter would for example pick a r6a.2xlarge node (4x8GiBs plus some deamonsets and OS overhead).
But why would practitioners even care about those instance families?
All we care is that we need X amount of CPU and 40ish GiBs of RAM.
This new generation of VMs could be powered by Firecracker like fargate is, or tradicional AL2/bottlerocket, where it still provides Root level access for those that need it.
I do understand that this more dynamic VM sizing would make it harder for AWS to better utilize their hosts, harder to bin pack VMs, etc. But in the large numbers, this would eventually negate the issue.
**Are you currently working around this issue?**
How are you currently solving this problem?
**Additional context**
Anything else we should know?
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
Contributor guide
Research direction
No repository files, tests, or implementation entry points are identified. Start by reviewing the EKS, Fargate, Karpenter, and Kubernetes scheduler context described in the issue; done would require an accepted product and implementation scope for resource-sized EKS capacity.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100