[Feature] Add support for registryPullQPS and registryPullBurst kubelet config
- Dominant language
- TypeScript
- Stars
- 2.1k
- Forks
- 395
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 13
Description
**Is your feature request related to a problem? Please describe.**
Clusters utilizing NAP/karpenter can be very tightly packed under normal conditions. During rapid scale-ups (think: Load tests, spikes in traffic), this causes new nodes to spin up and host new containers. If the nodes are large, this can mean hundreds of images need to be pulled and started on the node.
Currently, AKS uses the default value for registry pulls - 5 (QPS) and 10 (burst), coming from kubernetes code itself. There is no option to override this via kubelet config in [Custom config](https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration?tabs=linux-node-pools#supported-custom-configuration-parameters). If these values are too low, there is no way to override them.
In general, if a customer is using third-party/self-hosted registry and can tolerate high request count, the defaults can be pretty low. Also, even if QPS is too high, registries usually have rate limiting themselves.
**Describe the solution you'd like**
Extending the custom configuration parameters so registry QPS and registry burst are supported and passed onto the nodes.
**Describe alternatives you've considered**
Using a privileged daemonset that modifies kubelet config and restarts kubelet. However, this feels like a hack that also "messes" with the managed AKS platform. On top, additional work needs to be done to ensure the daemonset starts fast enough (before other containers are scheduled and exhaust the request limits) and it causes temporary blip in node readiness as kubelet restarts, slowing down initialization.
**Additional context**
I see two items for the same in the past, both closed due to inactivity. Is there a reason behind a "hard no" on supporting this or it is just not considered?
[4125](https://github.com/Azure/AKS/issues/4125)
[3138](https://github.com/Azure/AKS/issues/3138)
Contributor guide
Assessment
This issue has not been assessed yet.