[Feature] Allow to configure per-nodepool property to select outbound LB backenpool
- Dominant language
- TypeScript
- Stars
- 2.1k
- Forks
- 395
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 13
Description
**Is your feature request related to a problem? Please describe.**
We would like to be able to select which outbound IP(s) are used inside our distributed application.
**Describe the solution you'd like**
One very simple, elegant and effective possibility is to use one IP/list_of_IP/subnet per K8S nodepool.
The application schedule with IP(s) to use for witch workload/usage by scheduling pods on the proper nodepool, or pods could select tasks requiring a specific IP address to use in a message bus by filtering messages based on the pool they are running on.
Or many other possibilities.
To achieve that, you simply need
- a (or many) user provided LB on which you configure as many backendpool as you want with proper frontend and outbound rules
- a per nodepool property to configure which backendpool will be injected in the corresponding vmss "virtualMachineProfile.networkProfile.networkInterfaceConfigurations[0].ipConfigurations[0].loadBalancerBackendAddressPools"
The usage of these new nodepool properties could be subject/conditioned by a new outbound mode "userAssignedLoadbalancer", but it already work today and is just missing this trivial property plumbing between AKS orchestrator engine and the VMSS one. See below.
The functionality could be restricted to apiserver-vnet-integrated cluster to simplify things as the outbound path is not need by the nodes to contact their API servers in this case.
**Describe alternatives you've considered**
I have this setup running, but build/maintained in a very inelegant/fragile way:
- My cluster is configured with "userDefinedRouting" outbound mode (with an empty routing table) and is apiserver-vnet-integrated
- I deployed a NATGateway assigned to the cluster subnet for the deployment of the cluster
- I manually set "virtualMachineProfile.networkProfile.networkInterfaceConfigurations[0].ipConfigurations[0].loadBalancerBackendAddressPools" of all the underlying vmss with the wanted LB backendpools after the AKS deployment.
- Disassociate the NATgateway of the subnet : You now have per nodepool outbound IP(s) setup/full control.
All scaling up/down operations on the nodepools work without disruption.
Start/Stop of the K8S cluster work as intended too.
But for all upgrade/re-imaging/... of K8S (and so VMSS) you need to stop your workload, re-assing the NATgateway to your K8S subnet, do the upgrade, re-set the VMSS loadBalancerBackendAddressPools, disassociate the NAT gateway, restore the workload. Automatic upgrades are prohibited.
Contributor guide
Assessment
This issue has not been assessed yet.