Azure / Azure/AKS

[Feature] Add support for registryPullQPS and registryPullBurst kubelet config

Open
#5,863 5 comments 3 reactions 1 assignee Claimed by @allyford View on GitHub
feature-request kubelet
Dominant language
TypeScript
Stars
2.1k
Forks
395
Avg merge
2d 22h
Merged PRs (30d)
13

Description

**Is your feature request related to a problem? Please describe.**

Clusters utilizing NAP/karpenter can be very tightly packed under normal conditions. During rapid scale-ups (think: Load tests, spikes in traffic), this causes new nodes to spin up and host new containers. If the nodes are large, this can mean hundreds of images need to be pulled and started on the node.

Currently, AKS uses the default value for registry pulls - 5 (QPS) and 10 (burst), coming from kubernetes code itself. There is no option to override this via kubelet config in [Custom config](https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration?tabs=linux-node-pools#supported-custom-configuration-parameters). If these values are too low, there is no way to override them.

In general, if a customer is using third-party/self-hosted registry and can tolerate high request count, the defaults can be pretty low. Also, even if QPS is too high, registries usually have rate limiting themselves.

**Describe the solution you'd like**
Extending the custom configuration parameters so registry QPS and registry burst are supported and passed onto the nodes.

**Describe alternatives you've considered**
Using a privileged daemonset that modifies kubelet config and restarts kubelet. However, this feels like a hack that also "messes" with the managed AKS platform. On top, additional work needs to be done to ensure the daemonset starts fast enough (before other containers are scheduled and exhaust the request limits) and it causes temporary blip in node readiness as kubelet restarts, slowing down initialization.

**Additional context**
I see two items for the same in the past, both closed due to inactivity. Is there a reason behind a "hard no" on supporting this or it is just not considered?

[4125](https://github.com/Azure/AKS/issues/4125)
[3138](https://github.com/Azure/AKS/issues/3138)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.