aws / aws/aws-network-policy-agent

Proposed node-agent orphan-GC

Open
#620 1 comment 0 reactions 1 assignee Claimed by @parlakisik View on GitHub
enhancement
Dominant language
Go
Stars
75
Forks
68
Avg merge
2d 11h
Merged PRs (30d)
10

Description

A self-healing garbage-collection routine in the network-policy-agent that periodically reclaims orphaned eBPF programs/maps whose pod no longer exists on the node — instead of relying solely on the CNI DeletePodNp call to trigger cleanup.

Proposed approach:
- Extend the existing deleted-pods cleanup routine (startDeletedPodsCleanupRoutine, pkg/ebpf/bpf_client.go) with a periodic sweep.
- Enumerate pinned programs under /sys/fs/bpf/globals/aws/programs/*, parse each podIdentifier (reuse utils.GetPodIdentifierFromBPFPinPath, as recoverBPFState does).
- Resolve each podIdentifier to its pods via ingress/egressProgToPodsMap; if no host veth still exists (utils.GetHostVethName → netlink lookup) and it is not still referenced (existing isProgramReferenced
guard), call deleteBPFProbes(podIdentifier).
- Apply a grace period (reuse deletedPodsMinAge) to avoid racing pod ADD.

Why is this needed:

Today DeleteBPFProbes (the only unpin path) is invoked exclusively by the CNI DeletePodNp gRPC. When that call never arrives, the ingress+egress programs stay pinned in bpffs forever, growing unbounded per
node. This happens through no fault of the agent, via several triggers we've confirmed in production:
- Branch-ENI / Security-Groups-for-Pods + short-lived Job pods: ipamd DelNetwork must fetch the pod to release the branch ENI, but the completed Job pod is already gone from the K8s API (Failed to get pod
spec: ... Pod not found) → DEL aborts before network-policy cleanup.
- Node reboot / ipamd cold-start: tmpfs-backed checkpoint (/var/run/aws-node/ipam.json) is wiped → ipamd rebuilds from CRI → datastore: unknown pod on delete of pre-reboot pods.
- Partial attach failure (ingress pinned, egress load fails) on single-use Job podIdentifiers.

A node-side reconciler that keys off "pin present but pod/veth gone" makes the agent self-healing regardless of whether or why the CNI delete ever fires, closing all of these leak paths at once.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.