[Feature]: Expose RDMA NICs via CDI, as GPUs are today

Suggestion Description

What we do manually today

To give a container access to the RDMA NIC on an Instinct node, we hand-write the
following into every docker run command:

--device /dev/infiniband \
-v /usr/lib/x86_64-linux-gnu/libibverbs.so.1:/lib/x86_64-linux-gnu/libibverbs.so.1 \
-v /usr/lib/x86_64-linux-gnu/libionic.so.1:/usr/lib/x86_64-linux-gnu/libionic.so.1 \
-v /usr/lib/x86_64-linux-gnu/libibverbs/libionic-rdmav34.so:/usr/lib/x86_64-linux-gnu/libibverbs/libionic-rdmav34.so \
-v /etc/libibverbs.d:/etc/libibverbs.d:ro

The node's kernel already knows about the NIC. What is missing is on the container side:
RDMA is used through a userspace library, and unless that library is present inside the
container, ibv_devinfo cannot use the ionic RDMA device. The four mounts supply it, and
--device /dev/infiniband supplies the device file it opens.

What we would like instead

amd-ctk cdi generate already scans the host and writes /etc/cdi/amd.json describing
the device nodes and mount points a container needs for each GPU, so that
--device amd.com/gpu=all sets everything up. We would like the same treatment for the
NIC: have amd-ctk cdi generate also emit entries for the RDMA devices and the matching
host provider libraries, so the five flags above reduce to

docker run --device amd.com/gpu=all --device amd.com/rdma=all ...

Because the spec is generated on the host, the injected library would always match the
installed driver, which is the property we cannot get by baking it into an image.
Handling bnxt_re, ionic and mlx5 would cover the network stacks shipping with Instinct
systems today.

Why it matters

Every project ends up reimplementing this. ROCm/ATOM, for instance, ships
atom/mesh/scripts/docker_start.sh includes logic to detect the NIC type and assemble these mount flags. That logic belongs in the toolkit, where it can be generated from the host once, rather than being copied into each project's launch
scripts.

Operating System

Ubuntu 24.04

GPU

MI355X

ROCm Component

No response

Start at the amd-ctk cdi generate entry point and the existing GPU CDI generation path, then compare its output with atom/mesh/scripts/docker_start.sh. Done means generated CDI entries cover the RDMA device and provider-library mounts for bnxt_re, ionic, and mlx5, allowing the amd.com/rdma=all device request to replace the manual flags.