nestybox / nestybox/sysbox-mgr
MIG GPU passthrough via DRA/CDI (deviceMgr + MIG cap-node mknod)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 7
- Forks
- 23
- Avg merge
- 1h 49m
- Merged PRs (30d)
- 1
Description
Need
System containers need access to NVIDIA MIG GPUs allocated through Kubernetes DRA/CDI. When the NVIDIA driver is containerized (GPU-Operator), the per-slice MIG capability nodes under /dev/nvidia-caps never materialize on the host — they only exist inside the driver container at /run/nvidia/driver/dev. sysbox runs containers in a user namespace and bind-mounts device nodes from a host path, so without these nodes the container setup fails and the workload crashes.
Proposal
- Add a device manager (deviceMgr) that reconciles device discovery: a radix-tree of devicers (root/net/amd/intel/nvidia), per-container device-node creation under the sysbox lib dir, and DeviceMounts for bind-mounting into the container. The nvidia devicer reconciles container device paths against host roots {"/", "/run/nvidia/driver"} covering both direct /dev nodes and MIG caps under /dev/nvidia-caps.
- Add isMigCapNode() to detect char nodes under /dev/nvidia-caps/nvidia-cap* and mknodMigCapNode() to create a missing node on the host from its (major, minor) pair — idempotent, invoked from Discover() when capability reads fail for a MIG cap node. sysbox-mgr runs as root, so host mknod is allowed.
- Register the SetupDevices gRPC callback, remove devices on container teardown, and append device bind-mounts during mount request handling. Add go-immutable-radix dependency.
Benefits
Lets sysbox secure containers pass through NVIDIA MIG GPUs allocated via DRA/CDI, unblocking GPU-Operator-based MIG workspaces.
Part of a 3-repo feature (see sysbox-runc and sysbox-ipc PRs).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Discover(), the SetupDevices gRPC callback, container teardown, and mount request handling in sysbox-mgr. Verify how the nvidia devicer handles host roots and MIG capability nodes, then confirm that device discovery, host node creation, cleanup, and bind-mounts support DRA/CDI workloads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100