[Documentation]: add example integrating SriovNetworkNodePolicy/SriovNetwork (sriov-network-operator) for VF provisioning

Description of errors

Summary

k8s-network-device-plugin discovers and advertises already-existing VFs/PFs (via sysfs, e.g. pkg/utils/utils.go's GetVFconfigured/GetSriovVFcapacity), but it never creates them, there's no write path to sriov_numvfs anywhere in this repo. VF provisioning is currently left entirely to out-of-band steps (firmware/BIOS defaults, or a manual echo N > .../sriov_numvfs during node bootstrap), which isn't documented anywhere in this repo.

sriov-network-operator (k8snetworkplumbingwg/sriov-network-operator)[1] already solves this generically: its config-daemon writes sriov_numvfs via its always-loaded generic plugin (pkg/plugins/generic), independent of vendor, vendor-specific plugins (currently only Intel 8086 and Mellanox 15b3 in VendorPluginMap) are additive extras, not a requirement for basic VF creation. It also auto-renders NetworkAttachmentDefinitions via its SriovNetwork CRD.

Request

Add a documented, tested example (e.g. under docs/ or examples/) showing:

1. A SriovNetworkNodePolicy targeting AMD's NIC (vendor 1dd8), declaring numVfs and resourceName, and the required supported-nic-ids ConfigMap entry (supportedExtraNICs) since AMD's vendor/device IDs aren't in the default list.
2. A SriovNetwork CR replacing the manually-authored NetworkAttachmentDefinition (type: amd-host-device) with the operator-generated one (type: sriov), including how RESOURCE_PREFIX should be set to preserve the existing amd.com/nic resource name workloads already depend on.
3. Whether amd-host-device (cni/plugins/amd-host-device) is still needed/compatible in this flow, or whether sriov-cni fully replaces it — in particular, amd-host-device currently does IP-preservation on ADD/DEL (reads host interface IPs, builds static IPAM, restores state on teardown) that plain sriov-cni doesn't replicate, so switching CNI type isn't a transparent swap.
4. Clarify the interaction between resources.requests.amd.com/nic: N and Multus's k8s.v1.cni.cncf.io/networks annotation — reserving N VFs via resource requests does not by itself attach N interfaces; the network must be listed N times in the annotation. This is easy to get wrong when migrating and should be called out explicitly in whatever example gets added.

Why

Use case it unblocks:

Multi-tenant clusters running several high-throughput RCCL jobs per node, each hardware-isolated on its own VF, declaratively and survivable across node replacement.

[1] https://github.com/k8snetworkplumbingwg/sriov-network-operator

Possible Configuration:

# AMD

1. SriovNetworkNodePolicy — replaces AMD's device-plugin ConfigMap + (missing) VF creation

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: amd-nic-policy
  namespace: sriov-network-operator
spec:
  resourceName: nic                    # -> becomes amd.com/nic if RESOURCE_PREFIX=amd.com
  nodeSelector:
    feature.node.kubernetes.io/network-sriov.capable: "true"   # adjust to your node label
  numVfs: 8                                  # node-wide VF count this policy creates; size to your max concurrent demand
  nicSelector:
    vendor: "1dd8"                       # AMD/Pensando vendor ID, from the existing device-plugin config
    deviceID: "1002"                    # PF device ID (confirm against AMD's PCI ID docs)
  deviceType: netdevice             # not vfio-pci, matches RDMA/netdevice mode
  isRdma: true
  excludeTopology: false            # keep NUMA hints, same as ExcludeTopology: false today

## sriov-network-operator configuration
# values.yaml
supportedExtraNICs:
  - 'AMD_Pensando_DSC: "1dd8 1002 1003"'
And set the resource prefix so it matches your existing workload manifests:
# values.yaml
operator:
  resourcePrefix: amd.com

2. SriovNetwork — replaces the hand-written NetworkAttachmentDefinition

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
  name: amd-nic-network
  namespace: sriov-network-operator
spec:
  resourceName: nic                         # must match SriovNetworkNodePolicy.spec.resourceName
  networkNamespace: default         # namespace where the NAD gets created
  linkState: enable
  # ipam: '{"type":"whereabouts", ...}'   # add IPAM config; sriov-cni doesn't do the
                                                                 # dynamic host-IP-preservation amd-host-device did
---
apiVersion: v1
kind: Pod
metadata:
  name: workload-app
  labels:
    app: workload-app
  annotations:
    k8s.v1.cni.cncf.io/networks: amd-host-device-nad
spec:
  containers:
    - name: workload-container
      image: docker.io/rocm/roce-workload:ubuntu24_rocm7_rccl-J13A-1_anp-v1.1.0-4D_ainic-1.117.1-a-63
      command: ["sleep", "infinity"]
      securityContext:
        capabilities:
          add:
            - IPC_LOCK
            - NET_ADMIN
            - NET_RAW
      resources:
        requests:
          amd.com/gpu: 1
          amd.com/nic: 1
        limits:
          amd.com/gpu: 1
          amd.com/nic: 1

Attach any links, screenshots, or additional evidence you think will be helpful.

No response

Start by reading pkg/utils/utils.go and the existing AMD device-plugin configuration to understand the current VF and resource assumptions. Add a tested Kubernetes example covering SriovNetworkNodePolicy, SriovNetwork, RESOURCE_PREFIX, supportedExtraNICs, CNI compatibility, IP preservation, and repeated Multus network attachments; done means the migration behavior and resource-request semantics are explicit.