nvproxy needs GPU PCI topology without requiring RDMA devices
- Dominant language
- Go
- Stars
- 19.3k
- Forks
- 2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 264
Description
### Description
In a multi-GPU nvproxy-only sandbox(without --rdmaproxy), nvidia-smi topo -m fails:
```
nvidia-smi topo -m
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 CPU Affinity NUMA Affinity GPU NUMA ID
Failed to run topology matrix
```
Strace shows the relevant failure:
```
openat("/sys/bus/pci/devices/", O_DIRECTORY) = -1 ENOENT
```
The existing RDMA sysfs collector already includes GPU PCI leaves and ancestors for NCCL GPU-to-NIC distance calculation. However, this only runs when an actual /dev/infiniband/uverbsN device is present:
```
RDMAEnabled = conf.RDMAProxy && specHasUverbsDevice(spec)
```
Could GPU PCI topology collection and rendering be decoupled from RDMA, so nvproxy can expose a read-only /sys/devices/pci... and /sys/bus/pci/devices topology independently, while reusing the RDMA snapshot when one is available?
### Steps to reproduce
docker run --rm --gpus=all --runtime=runsc pytorch/pytorch:2.13.0-cuda13.0-cudnn9-runtime nvidia-smi topo -m
### runsc version
```shell
```
### docker version (if using docker)
```shell
```
### uname
_No response_
### kubectl (if using Kubernetes)
```shell
```
### repo state (if built from source)
_No response_
### runsc debug logs (if available)
```shell
```
Contributor guide
Research direction
Start with the existing RDMA sysfs collector and the RDMAEnabled/specHasUverbsDevice decision described in the issue. Reproduce with the provided docker run command, then trace how PCI leaves and ancestors are exposed; done means nvidia-smi topo -m works in an nvproxy-only sandbox while reusing the RDMA snapshot when available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go, linux
- Domain
- infrastructure, operating-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100