microsoft / microsoft/retina

Network traffic stops after Retina DaemonSet restart on AKS with Cilium

Open
#1,804 5 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
3.2k
Forks
304
Avg merge
1d 19h
Merged PRs (30d)
78

Description

[cilium.log](https://github.com/user-attachments/files/21615071/cilium.log)
[retina.log](https://github.com/user-attachments/files/21615072/retina.log)

We are running an AKS cluster with Cilium installed and would like to use Retina for network observability.

However, after restarting the Retina DaemonSet (e.g., due to a configuration update), all network traffic in the AKS cluster stops. The only way to restore connectivity is by restarting the Cilium agent on the affected nodes.

Additionally, after Retina restarts, the following warning repeatedly appears in the Cilium logs:

`level=warning msg="Detected unexpected endpoint BPF program removal. Consider investigating whether other software running on this machine is removing Cilium's endpoint BPF programs. If endpoint BPF programs are removed, the associated pods will lose connectivity and only reinstating the programs will restore connectivity." count=12 subsys=daemon`

This makes Retina a no-go for us in production.

**Steps to reproduce:**
- Deploy Retina in an AKS cluster with Cilium.
- Restart the Retina DaemonSet (e.g., kubectl rollout restart daemonset retina).
- Observe that network traffic stops.
- Check Cilium logs for warnings about BPF program removal.
- Restart the Cilium agent on the affected nodes to restore connectivity.

**Expected behavior:**
Restarting Retina should not interfere with Cilium’s BPF programs or disrupt network traffic.

**Environment:**
- AKS
- Cilium version: 1.17.4
- Retina version: v0.0.36
- Kubernetes version: 1.32.3

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.