aws / aws/aws-network-policy-agent

"unable to create/update map entry and ret -1 and err argument list too long" error resulting in kubelet probe failures to pods

Open
#484 5 comments 0 reactions 1 assignee Claimed by @zac-nixon View on GitHub
bug
Dominant language
Go
Stars
76
Forks
69
Avg merge
2d 11h
Merged PRs (30d)
10

Description

**What happened**:

We observed kubelet probe failures from a small number of pods on a single node over a period of a day that resulted in the pods being unable to start, this did not impact any currently live pods. These probes repeatedly failed with `context deadline exceeded (Client.Timeout exceeded while awaiting headers)`.

This impacted a single node and looks to have been scoped to pods being part of the same `Deployment`. I was able to `exec` into one of these pods and obtain outbound connectivity so this was either only impacting ingress traffic (untested) or only impacting ingress traffic from kubelet -> pod for probes.

We managed to track each of these failures back to the same errors in the `ebpf-sdk.log` file as follows, which repeated a number of times on each failure:

```
{"level":"error","ts":"2025-11-14T00:06:57.823Z","caller":"maps/loader.go:286","msg":"unable to create/update map entry and ret -1 and err argument list too long"}
```

**Attach logs**

I can provide additional logs and/or configuration privately if required but only at `info` level as that is what we currently log at to avoid policy decision logs at `ACCEPT` level being logged due to https://github.com/aws/aws-network-policy-agent/issues/467.

I've also only captured the contents of `/var/log/aws-routed-eni` as the log script from https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/troubleshooting.md#collecting-node-level-tech-support-bundle-for-offline-troubleshooting does not currently work on Bottlerocket which we run.

Please let me know if you'd like me to email these to awscni-triage@amazon.com.

**What you expected to happen**:

Pod probes to complete successfully so that the pod can start.

**How to reproduce it (as minimally and precisely as possible)**:

I don't have a reproduction as this has only happened on a single node so far and has not recurred again after terminating the node.

**Anything else we need to know?**:

**Environment**:
- Kubernetes version (use `kubectl version`): `v1.33.5-eks-3cfe0ce`
- CNI Version: `v1.20.3`
- Network Policy Agent Version: `v1.2.6`
- OS (e.g: `cat /etc/os-release`): `Bottlerocket OS 1.50.0 (aws-k8s-1.33)`
- Kernel (e.g. `uname -a`): `6.12.53 #1 SMP PREEMPT_DYNAMIC Tue Oct 28 18:05:29 UTC 2025 x86_64 GNU/Linux`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.