aws / aws/containers-roadmap

Ability to set VPC CNI configuration variables per instance type

Open
#1,834 4 comments 38 reactions 0 assignees View on GitHub
EKS EKS Networking Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
What do you want us to build?

I would like to be able to set VPC CNI configuration variables https://github.com/aws/amazon-vpc-cni-k8s#cni-configuration-variables per instance type using userdata (or another mechanism). When the VPC CNI pod starts on the host, I would like it to check for any variables I've set on the node and use those values instead of the global values set via the VPC CNI.

For example, MINIMUM_IP_TARGET and WARM_IP_TARGET configuration for the VPC CNI is a global setting and is applied to all instances deployed via kubernetes, regardless of whether these nodes are deployed via mechanisms like the Kube API, CloudFormation, EKSCTL, Cluster Autoscaler or Karpenter etc.

I would like to have the ability to set MINIMUM_IP_TARGET and WARM_IP_TARGET for either specific instance types deployed by CAS or Karpenter, for example.

**Which service(s) is this request for?**
This could be EKS, Fargate

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

I am trying to solve the over allocation of IPs when using MINIMUM_IP_TARGET for all instance types. Say I have a batch workload which scale up, I want use 60 IPs for large instance types and a MINIMUM_IP_TARGET. But for smaller nodes in other node groups that manage other types of pods, will use this value and consume unnecessary IPs.

By setting these values on the node via userdata, I can control different instance types MINIMUM_IP_TARGET variable value. Different instance types can run different numbers of pods, therefore the MINIMUM_IP_TARGET can and should be different. Smaller instance types could have a smaller MINIMUM_IP_TARGET value whereas larger instance types could have a larger MINIMUM_IP_TARGET value.

It is currently hard to set these values per instance type using custom code as these are global settings applied via teh VPC CNI.

**Are you currently working around this issue?**

We're not but we are experimenting with init containers to see I we can set values ourselves

**Additional context**
Anything else we should know?

I logged a similar feature request [here](https://github.com/aws/amazon-vpc-cni-k8s/issues/2072) but was told to log it within the containers roadmap.

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

No repository files, tests, or implementation entry points are identified. Start by reviewing the VPC CNI configuration variables and the related amazon-vpc-cni-k8s issue; done means node- or instance-specific values can override the global settings for variables such as MINIMUM_IP_TARGET and WARM_IP_TARGET.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.