[EKS] [request]: Exclude pods using the host network from the --max-pods calculation
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
Exclude pods using the host network from the `--max-pods` calculation.
When calculating `--max-pods`, it is inefficient to fix the number of host network pods at 2.
If using more host network pods other than `aws-node`/`kube-proxy`, available IPs are wasted due to `--max-pods` restrictions.
I expect that when calculating `--max-pods` it will only target pods with private ips.
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
When calculating `--max-pods`(https://github.com/awslabs/amazon-eks-ami/blob/master/files/eni-max-pods.txt), obtain max eni and private ip per eni through the type of node instance and then calculate the total number of ips.
~~~
# Mapping is calculated from AWS EC2 API using the following formula:
# * First IP on each ENI is not used for pods
# * +2 for the pods that use host-networking (AWS CNI and kube-proxy)
#
# # of ENI * (# of IPv4 per ENI - 1) + 2
#
# https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI
~~~
However, as the number of pods using the host network increases, the number of pods using private IPs decreases.
When I use t3.medium type to node instance, which can use 3 ENIs and 6 IPs per ENI. According to [eni-max-pods.txt](https://github.com/awslabs/amazon-eks-ami/blob/master/files/eni-max-pods.txt), a maximum 17 pods can be placed on this node.
> ignored aws-node warm pool settings
**Case-1 : 2 host-network pods, 15 deployment pods**
~~~
NAMESPACE NAME READY STATUS RESTARTS AGE
game-2048 deployment-2048-764544cd4f-1 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-2 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-3 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-4 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-5 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-6 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-7 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-8 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-9 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-10 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-11 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-12 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-13 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-14 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-15 1/1 Running 0 16m
kube-system aws-node-945nv 1/1 Running 0 16m
kube-system kube-proxy-pf6k5 1/1 Running 0 16m
~~~
~~~
Node ENI/Private IP status
- eni0(6 IPs) : 1 Primay IP + 5 Private IPs
- eni1(6 IPs) : 1 Primay IP + 5 Private IPs
- eni2(6 IPs) : 1 Primay IP + 5 Private IPs
~~~
Can deployed a total of 17 pods, calculated with `--max-pods`, and are using all available IPs.
**Case-2: 4 host-network pods, 13 deployment pods**
~~~
NAMESPACE NAME READY STATUS RESTARTS AGE
game-2048 deployment-2048-764544cd4f-1 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-2 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-3 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-4 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-5 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-6 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-7 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-8 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-9 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-10 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-11 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-12 1/1 Running 0 16m
game-2048 deployment-2048-764544cd4f-13 1/1 Running 0 16m
kube-system aws-node-945nv 1/1 Running 0 16m
kube-system kube-proxy-pf6k5 1/1 Running 0 16m
kube-system efs-csi-controller-6ddf8d6d64-5vbvv 3/3 Running 0 16m
kube-system efs-csi-node-spgfj 3/3 Running 0 16m
~~~
~~~
Node ENI/Private IP status
- eni0(6 IPs) : 1 Primay IP + 5 Private IPs
- eni1(6 IPs) : 1 Primay IP + 5 Private IPs
- eni2(6 IPs) : 1 Primay IP + 3 Private IPs
~~~
Can deployed a total of 17 pods, calculated with `--max-pods`, and left 2 IPs and can't using it.
**Are you currently working around this issue?**
How are you currently solving this problem?
**Additional context**
Anything else we should know?
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
Contributor guide
Research direction
Start with files/eni-max-pods.txt and the issue's EC2 ENI and private-IP formula. Trace how the --max-pods mapping is calculated, then verify that host-network pods beyond aws-node and kube-proxy do not consume private-IP capacity; done means the t3.medium example can use the remaining IPs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100