gpustack / gpustack/gpustack-operator

todo: a bond or VLAN carrying Soft-RoCE can never publish rdma or a verdict

Open
#180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/devicemanager todo
Dominant language
Go
Stars
4
Forks
7
Avg merge
3h 9m
Merged PRs (30d)
213

Description

What is left undone

readInterface returns a virtual interface — anything under devices/virtual/net: a bond, a VLAN, a
veth — before resolveRDMA runs (pkg/devicemanager/detector/network.go; the virtual branch sits
above both the device symlink resolution and the RDMA lookup). Such an interface therefore never
carries an rdma record and never carries a link verdict, so it can neither publish nor refresh
feature.gpustack.ai/rdma.capable.

That is correct for what the branch was written for. It is wrong for software RDMA: rxe
(Soft-RoCE) and siw bind an RDMA device to an ordinary netdev, and nothing stops that netdev from
being a bond or a VLAN. On such a node the RDMA endpoint exists, works, and is invisible to this
reader.

Why reordering does not fix it

Every path to an RDMA device in this reader starts from a PCI device directory, and a bond, a VLAN or
a veth has no device symlink to start from. The reverse /sys/class/infiniband/* index is keyed the
same way. So the early return is not the cause: the complement of that branch is unreachable, not
merely skipped, and moving the return would change nothing.

Closing this means giving the RDMA side an index keyed by netdev name rather than by PCI address.
/sys/class/infiniband/<device>/ports/*/gid_attrs/ndevs/* names interfaces directly, so the lookup
exists; what does not exist is a consumer of it for interfaces that have no PCI device.

What does not close this
  • A fixture whose virtual interface carries an rdma record. That proves the struct can hold one;
    it does not prove this reader can produce one. triggersDetect already exempts only virtual
    interfaces with no RDMA record — which reads as if such records existed, and the code comment on
    that branch says so deliberately. A fixture asserting today's behaviour would convert that note into
    apparent coverage.
  • A line in the design record. There is none today (below), and adding one would record the limit
    without making the endpoint discoverable.
Why it needs an issue rather than a spec line

The limit survives in exactly one place: the code comment on the virtual branch. It is not in the
merged design record — grep -iE 'rxe|soft-roce|siw|software rdma' over
specs/2026-09-02-devices-nic-rdma-topology.md returns 0 — and its only other landing place was an
open review thread on #162, which is merged.

Scope

No environment available to this project runs rxe or siw, so this is a gap in reachable behaviour
rather than an observed failure. That is why it is a todo and not a bug: the work is well defined,
and the trigger for doing it is a user who runs software RDMA over a bonded or tagged interface.

Introduced with the NIC/RDMA topology work in #162.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/devicemanager/detector/network.go and trace readInterface, resolveRDMA, and triggersDetect; review the referenced sysfs path under /sys/class/infiniband//ports//gid_attrs/ndevs/*. Add coverage for software RDMA on a bond or VLAN without a PCI device, then verify the interface can publish and refresh the rdma capability and link verdict.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.