kubernetes-sigs / kubernetes-sigs/aws-load-balancer-controller

Controller Cannot Resolve ENI for IPv6 only Pods

Open
#4,643 4 comments 0 reactions 0 assignees View on GitHub
kind/bug lifecycle/stale
Dominant language
Go
Stars
4.3k
Forks
1.6k
Avg merge
1d 15h
Merged PRs (30d)
6

Description

**Bug Description**

When using IPv6-only pods with the Security Groups for Pods feature enabled (`ENABLE_POD_ENI = true` on VPC CNI) and the controller configured to manage backend security groups (`alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"`), the controller fails with:

```
{"level":"error","ts":"","msg":"Requesting network requeue due to error from ReconcileForPodEndpoints","tgb":{"name":"" ,"namespace":""},"error":"cannot resolve pod ENI for pods: [ ...]"}
```

I suspect the root cause is in [resolveViaPodENIAnnotation](https://github.com/kubernetes-sigs/aws-load-balancer-controller/blob/acfc9ac2c4d5b538dc5ec5c3930c32df24761f6a/pkg/networking/pod_eni_info_resolver.go#L225-L262). The function matches the pod's ENI annotation by comparing `podENIInfo.PrivateIP == pod.PodIP` (line 230). However, the `PodENIInfo.PrivateIP` field is always an IPv4 address (populated from the `vpc.amazonaws.com/pod-eni` annotation), but for IPv6-only pods, `pod.PodIP` is an IPv6 address. This comparison will never succeed, so the branch ENI ID is never discovered.

The cascaded fallback to resolveViaNodeENIs also fails because the pod's IPv6 address lives on the branch ENI, not on the node's trunk ENI, so no match is found there either. Both resolution methods fail, leaving pods unresolved.

**Steps to Reproduce**

- Step-by-step guide to reproduce the bug:
- Configure an EKS cluster with a dualstack VPC/subnet for private workloads. Deploy EKS cluster vpc cni with IPv6 (https://docs.aws.amazon.com/eks/latest/userguide/cni-ipv6.html)
- Enable Security Groups for Pods (branch ENIs) via the VPC CNI plugin (`ENABLE_POD_ENI=true`).
- Create a SecurityGroupPolicy targeting workload with a test security group
- Deploy a workload (Deployment/Service) with pods that receive IPv6 addresses and a dedicated ENI. The pods should receive the `vpc.amazonaws.com/pod-eni` annotation.
- Configure the AWS Load Balancer Controller with backend security group management enabled.
- Create an Ingress resource of type ALB targeting these pods. Ingress should also be configured for backend security group management (`alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"`)
- Manifests applied while reproducing the issue:
```
apiVersion: vpcresources.k8s.aws/v1beta1
kind: SecurityGroupPolicy
metadata:
name: sgpp-repro-policy
namespace: ipv6-sgpp-repro
spec:
podSelector:
matchLabels:
app: sgpp-repro
securityGroups:
groupIds:
- sg-xxxxxxxxxxxxxxxxx
```

```
apiVersion: apps/v1
kind: Deployment
metadata:
name: sgpp-repro
namespace: ipv6-sgpp-repro
spec:
replicas: 2
selector:
matchLabels:
app: sgpp-repro
template:
metadata:
labels:
app: sgpp-repro
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
protocol: TCP
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
```

```
apiVersion: v1
kind: Service
metadata:
name: sgpp-repro-svc
namespace: ipv6-sgpp-repro
spec:
internalTrafficPolicy: Cluster
ipFamilies:
- IPv6
ipFamilyPolicy: SingleStack
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: sgpp-repro
sessionAffinity: None
type: ClusterIP
```

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sgpp-repro-ingress
namespace: ipv6-sgpp-repro
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/ip-address-type: dualstack
alb.ingress.kubernetes.io/subnets: subnet-aaaaaaa,subnet-bbbbbbb
alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"
spec:
rules:
- host: sgpp-repro.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: sgpp-repro-svc
port:
number: 80
```
- Controller logs/error messages while reproducing the issue:

```
{"level":"info","ts":"2026-03-25T14:49:55Z","msg":"Requeue for networking requeue","tgb":{"name":"","namespace":""}}
{"level":"error","ts":"2026-03-25T14:50:10Z","msg":"Requesting network requeue due to error from ReconcileForPodEndpoints","tgb":{"name":"", "namespace":""},"error":"cannot resolve pod ENI for pods: [/ ...]"}
```

**Expected Behavior**

The controller should successfully resolve the branch ENI for IPv6-only pods using the `vpc.amazonaws.com/pod-eni` annotation, regardless of whether the pod IP is IPv4 or IPv6. Then configure the pod security group for ingress from the ALB.

**Actual Behavior**

The controller fails to resolve the ENI for IPv6-only pods that use Security Groups for Pods. This prevents Ingress/Service resources from being reconciled.

**Regression**
Was the functionality working correctly in a previous version ? I don't think so.
I can't say I have tested previous versions but I suspect this issue has been around for a while.

**Current Workarounds**

I can manage the security group rules manually and allow ingress for the ALB directly but would very much prefer to have the controller do it for me.

**Environment**
- AWS Load Balancer controller version: 1.13.4
- Kubernetes version: 1.34
- Using EKS (yes/no), if so version?: yes (1.34)
- Using Service or Ingress: Ingress
- AWS region: us-west-2
- How was the aws-load-balancer-controller installed: helm
- `aws-load-balancer-controller kube-system 1 2026-02-17 16:58:59.486094771 +0000 UTC deployed aws-load-balancer-controller-1.13.4 v2.13.4`
- If helm was used then please show output of `helm -n get values `:
```
USER-SUPPLIED VALUES:
additionalLabels:
...
clusterName:
podLabels:
...
region: us-west-2
resources:
limits:
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
serviceAccount:
annotations:
eks.amazonaws.com/role-arn:
vpcId: vpc-XXXXXXXXXXX
```
- Current state of the Controller configuration:
- `kubectl -n describe deployment aws-load-balancer-controller`:
```
Name: aws-load-balancer-controller
Namespace: kube-system
CreationTimestamp: Tue, 17 Feb 2026 09:59:00 -0700
Labels: app.kubernetes.io/instance=aws-load-balancer-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=aws-load-balancer-controller
app.kubernetes.io/version=v2.13.4
helm.sh/chart=aws-load-balancer-controller-1.13.4
helm.toolkit.fluxcd.io/name=aws-lb-controller
Annotations: deployment.kubernetes.io/revision: 1
meta.helm.sh/release-name: aws-load-balancer-controller
meta.helm.sh/release-namespace: kube-system
Selector: app.kubernetes.io/instance=aws-load-balancer-controller,app.kubernetes.io/name=aws-load-balancer-controller
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels:
...
Annotations: prometheus.io/port: 8080
prometheus.io/scrape: true
Service Account: aws-load-balancer-controller
Containers:
aws-load-balancer-controller:
Image: public.ecr.aws/eks/aws-load-balancer-controller:v2.13.4
Ports: 9443/TCP, 8080/TCP
Host Ports: 0/TCP, 0/TCP
Args:
--cluster-name=
--ingress-class=alb
--aws-region=us-west-2
--aws-vpc-id=vpc-XXXXXXXXXXXXXX
Limits:
memory: 256Mi
Requests:
cpu: 100m
memory: 128Mi
Liveness: http-get http://:61779/healthz delay=30s timeout=10s period=10s #success=1 #failure=2
Readiness: http-get http://:61779/readyz delay=10s timeout=10s period=10s #success=1 #failure=2
Environment:
Mounts:
/tmp/k8s-webhook-server/serving-certs from cert (ro)
Volumes:
cert:
Type: Secret (a volume populated by a Secret)
SecretName: aws-load-balancer-tls
Optional: false
Priority Class Name: system-cluster-critical
Node-Selectors:
Tolerations:
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available True MinimumReplicasAvailable
OldReplicaSets:
NewReplicaSet: aws-load-balancer-controller-5fbf89f968 (2/2 replicas created)
Events:
```
- Current state of the Ingress/Service configuration:
- `kubectl describe ingressclasses`:
```
Name: alb
Labels: app.kubernetes.io/instance=aws-load-balancer-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=aws-load-balancer-controller
app.kubernetes.io/version=v2.13.4
helm.sh/chart=aws-load-balancer-controller-1.13.4
helm.toolkit.fluxcd.io/name=aws-lb-controller
Annotations: meta.helm.sh/release-name: aws-load-balancer-controller
meta.helm.sh/release-namespace: kube-system
Controller: ingress.k8s.aws/alb
Events:
```
- `kubectl -n describe ingress `
```
Name:
Labels:
Namespace:
Address:
Ingress Class: alb
Default backend:
TLS:
SNI routes
Rules:
Host Path Backends
---- ---- --------

/ :8180 ([]:8180,[]:8180,[]:8180)
Annotations: alb.ingress.kubernetes.io/backend-protocol: HTTPS
alb.ingress.kubernetes.io/group.name: shared-ingress
alb.ingress.kubernetes.io/group.order: 10
alb.ingress.kubernetes.io/healthcheck-path: /health/ready
alb.ingress.kubernetes.io/healthcheck-port: 9000
alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
alb.ingress.kubernetes.io/listen-ports: [{"HTTPS": 443}]
alb.ingress.kubernetes.io/load-balancer-name:
alb.ingress.kubernetes.io/target-type: ip
Events:
```
- `kubectl -n describe svc `
```
Name: -service
Namespace:
Labels: app=
Annotations:
Selector: app.kubernetes.io/instance=,
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv6
IP:
IPs:
Port: https 8180/TCP
TargetPort: 8180/TCP
Endpoints: []:8180,[]:8180,[]:8180
Port: management 9000/TCP
TargetPort: 9000/TCP
Endpoints: []:9000,[]:9000,[]:9000
Session Affinity: None
Internal Traffic Policy: Cluster
Events:
```

**Possible Solution (Optional)**

I'm not super familiar with the code base but I think this could be fixed by:

1. Expand `PodInfo` to carry all pod IPs including any IPv6 addresses.
2. Populate `PodInfo` with all pod IPs in `buildPodInfo`
3. Change `resolveViaPodENIAnnotation` to match against all pod IPs in `PodInfo`

**Contribution Intention (Optional)**

- [X] Yes, I'm willing to submit a PR to fix this issue
- [ ] No, I cannot work on a PR at this time

**Additional Context**

Contributor guide

Open the contributing guide

Research direction

Start in pkg/networking/pod_eni_info_resolver.go at resolveViaPodENIAnnotation, then trace ReconcileForPodEndpoints and the vpc.amazonaws.com/pod-eni annotation handling. Reproduce with the IPv6-only pod, Security Groups for Pods, and backend security-group settings described in the issue. Done means the controller resolves the branch ENI and reconciles the Ingress without the cannot resolve pod ENI error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
backend, cloud, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.