NVIDIA / NVIDIA/gpu-operator

gpu-operator daemonsets are terminated when the NFD device labels include PCI device IDs

Open
#1,469 1 comment 0 reactions 1 assignee View on GitHub

@JunAr7112 is already working on this.

Since Sep 15, 2026.

feature lifecycle/frozen
Dominant language
Go
Stars
2.9k
Forks
552
Avg merge
2d 4h
Merged PRs (30d)
90

Description

Currently the gpu-operator relies on the Node Feature Discovery operator to apply thefeature.node.kubernetes.io/pci-10de.present=true node label for any nodes that contain NVIDIA hardware.
If the gpu-operator daemonsets are already deployed and the cluster admin modifies the sources.pci.deviceLabelFields property to add the device PCI ID in the NodeFeatureDiscovery CR, this will cause an immediate termination of the gpu-operator stack.

apiVersion: nfd.openshift.io/v1
kind: NodeFeatureDiscovery
metadata:
  name: openshift-nfd
  namespace: openshift-nfd
spec:
  workerConfig:
    configData: |
      core:
        sleepInterval: 60s
      sources:
        pci:
          deviceClassWhitelist:
            - "0200"
            - "03"
            - "12"
          deviceLabelFields:
            - "vendor"
            - "device"         # <----This blocks OR terminates the NVIDIA gpu-operators stack

Since the NodeFeatureDiscovery is a shared global resource it would be a best practice to bring the ownership of gpu-operator driver rollout into a custom node label applied by a NodeFeatureRule.

You already apply the node label nvidia.com/gpu.present=true after feature.node.kubernetes.io/pci-10de.present=true is detected could nvidia.com/gpu.present=true be elevated to a NodeFeatureRule that is applied by the gpu-operator automatically after installation?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.