[Feature] Cilium request values
- Dominant language
- TypeScript
- Stars
- 2.1k
- Forks
- 395
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 13
Description
Azure CNI Managed by Cilium currently lacks request values for the agent pods and cilium-operator pods. The agent pods request 100Mi of memory based on the request value of the init-container clean-cilium-state, while all other containers do not have specified request values. This situation can lead to OOMKilled events on nodes if the request value is too low for the cluster's usage. Without a limit value set for the cilium-agent, it may consume more memory than requested, potentially causing other workloads to be terminated by the node during an OOMKilled event.
In addition to Cilium, other workloads like acns-security also lack request values, leading to resource under-provisioning for system components. This can result in the termination of system or user pods and can also hinder autoscaling mechanisms such as Node Autoprovision or Cluster Autoscaler.
To address these issues, we have temporarily increased the System node pool SKU and minimum count to accommodate the unrequested resources. However, this mitigation is not suitable for User node pools, as user workloads are dynamic and will utilize unrequested resources as needed during operation.
As a second mitigation we are using a LimitRange resource in the kube-system namespace:
```yaml
apiVersion: v1
kind: LimitRange
metadata:
name: defaultresources
namespace: kube-system
spec:
limits:
- default:
memory: 500Mi
defaultRequest:
memory: 500Mi
cpu: 250m
type: Container
```
For a long-term solution, we are requesting that the cilium-agent be assigned appropriate request values. To accommodate the dynamic nature of resource requirements, it would be beneficial to leverage Vertical Pod Autoscaler (VPA) for managing resource values, as outlined in issue #2125. We also propose the inclusion of Cilium in this optimization effort.
Contributor guide
Assessment
This issue has not been assessed yet.