Azure / Azure/AKS

[Feature] Support explicit Mellanox or MANA adapter selection for AKS node pools

Open
#5,931 2 comments 0 reactions 2 assignees Claimed by @allyford View on GitHub
feature-request nodepools
Dominant language
TypeScript
Stars
2.1k
Forks
395
Avg merge
2d 22h
Merged PRs (30d)
13

Description

**Is your feature request related to a problem? Please describe.**

AKS node pools using Accelerated Networking VM sizes that are eligible for Microsoft Azure Network Adapter (MANA) can be placed on either MANA or Mellanox hardware. Specialized workloads can have different compatibility or validation requirements for the two VF implementations.

There is currently no supported AKS node-pool property to:

- Request MANA or Mellanox placement when creating a node pool.
- Expose the actual adapter implementation as an AKS-managed Kubernetes node label or condition.
- Ensure that Cluster Autoscaler scale-out nodes use the required adapter implementation.
- Perform a controlled migration from Mellanox to MANA, or from MANA to Mellanox.

Custom Azure tags and AKS node-pool labels only describe intent; they do not participate in Compute host placement. Detecting the VF driver after provisioning and manually labeling individual nodes is not reliable for autoscaling, node repair, node-image upgrades, or node replacement.

**Describe the solution you'd like**

Provide a supported AKS mechanism with the following capabilities:

1. Allow a node pool to request an Accelerated Networking adapter implementation, for example `auto`, `mana`, or `mellanox`, subject to documented SKU, region, and capacity availability.
2. Expose the adapter actually assigned to every node through an AKS-managed, immutable Kubernetes label or node condition, such as `kubernetes.azure.com/network-adapter=mana|mellanox`.
3. Make Cluster Autoscaler and other AKS scaling operations honor the requested adapter requirement. If matching capacity is unavailable, surface a clear provisioning or scaling error rather than silently using a different implementation.
4. Provide a supported migration workflow. If an in-place change is not possible, AKS could perform a controlled rolling replacement into a new node pool with drain, surge, and PodDisruptionBudget behavior.
5. Document the expected behavior during node repair, reimage, Kubernetes upgrade, node-image upgrade, and infrastructure maintenance.
6. Document whether fallback to another adapter is permitted and provide an option to fail closed when a workload has a strict adapter requirement.

**Describe alternatives you've considered**

- Applying Azure tags such as `nic-intent=mellanox` to the AKS node pool or VMSS. These tags do not control physical host placement.
- Creating a dedicated node pool with a specific MANA-eligible VM size. Nodes of the same size can still be placed on different adapter implementations.
- Inspecting PCI devices and VF drivers after provisioning, then manually adding Kubernetes labels. This can support fixed-capacity testing but is not reliable for automatic scale-out or node replacement.
- Keeping fixed, individually verified Mellanox capacity as a temporary workaround.
- Updating applications to support both MANA and Mellanox. This remains the preferred long-term compatibility approach, but an AKS-managed discovery and migration signal would make validation and rollout safer.

**Additional context**

On Linux, the routable `eth0` interface normally uses `hv_netvsc`, while the child Accelerated Networking VF uses `mana`, `mlx4_en`, or `mlx5_core`. The adapter must therefore be identified from PCI and driver evidence rather than interface speed, VMSS instance number, kernel version, Azure tags, or user-supplied node labels.

This request is for a supported AKS and Azure Compute placement and discovery contract. It is not a request to modify the AKS-managed VMSS directly or to treat a custom tag as proof of hardware placement.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.