aws / aws/containers-roadmap

[EKS] [request]: Auto Mode support for Security Groups per Pod

Open
#2,632 5 comments 19 reactions 0 assignees View on GitHub
EKS EKS Auto Mode Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
I would love to see Auto Mode support for Security Groups Per Pod.

**Which service(s) is this request for?**
EKS (Auto Mode)

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
We use SGPP on Fargate-scheduled workloads today to manage "north-south" access to AWS resources in our VPC that support Security Group network rule management, such as AWS RDS or VPC Interface Endpoint. (Since Fargate doesn't support k8s NetworkPolicy, we also use Security Groups to manage "east-west" network traffic in our EKS Cluster between Pods scheduled on Fargate.)

As described in this document (https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html), we create a SecurityGroupPolicy for Pods that we schedule to Fargate. I've observed that when we attempt to schedule a Pod on Auto Mode-managed compute, the resource request for "vpc.amazonaws.com/pod-eni" cannot be met by Karpenter. And indeed the AWS docs are clear that Auto Mode does not support SGPP (https://docs.aws.amazon.com/eks/latest/userguide/auto-networking.html)

With our need to apply Security Groups to our workloads, the inability to apply SGPP in Auto Mode might lead us towards the securityGroupSelectorTerms/podSecurityGroupSelectorTerms config described here (https://docs.aws.amazon.com/eks/latest/userguide/create-node-class.html). However this seems to imply we'd need to maintain a NodeClass and NodePool per workload, in addition to the taint/toleration/nodeAffinity/nodeSelector config necessary to schedule each workload appropriately on its own Node.

**Are you currently working around this issue?**
Currently, this issue has led us to stay on Fargate for Clusters where we deploy multiple workloads. We would like to migrate our use of Fargate to EKS Auto Mode.

**Additional context**
I'll be the first to admit that I'm still maturing on EKS and k8s usage, so, would welcome insight into workarounds or better approaches. One ultimate solution would be to outgrow the use of Fargate and Auto Mode, but we're not quite there yet, and I'd suppose there are other AWS customers in this situation.

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked EKS documentation for Security Groups for Pods, Auto Mode networking, and NodeClass configuration. Clarify the supported design for applying per-pod security groups in Auto Mode; done means an agreed product approach or documented support path, not a change identified in this roadmap issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, infrastructure, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.