canonical / canonical/nvidia-core22

NVIDIA runtime doesn't show up in Docker snap on Ubuntu Core 22

Open
#18 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
1
Forks
4
PR merge metrics
No merged PRs in 30d

Description

I have installed Ubuntu Core 22 on a pc that has a NVIDIA 3090 GPU. Previously, this worked fine with Docker when I had Ubuntu server installed but since moving to Ubuntu Core 22, I have not been able to see the nvidia runtime in the docker snap. Also, the output of `lspci` (included below) seems to indicate that it hasn't recognized it as a 3D Controller. Since I don't see any issues, perhaps I am missing a step?

### Some relevant info

Output of `nvidia-core22.smi`

```
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3090 Off | 00000000:21:00.0 Off | N/A |
| 0% 37C P8 29W / 420W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+
```

Output of `ls -la /snap/nvidia-core22/current/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.535.161.08`

```
-rw-r--r-- 1 root root 1946840 Mar 5 22:13 /snap/nvidia-core22/current/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.535.161.08
```

Output of `sudo docker info | grep Runtime`

```
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
```

Snap's installed Ubuntu Core 22
```
Name Version Rev Tracking Publisher Notes
core 16-2.61.2 16928 latest/stable canonical✓ core
core20 20240416 2318 latest/stable canonical✓ base
core22 20240408 1380 latest/stable canonical✓ base
docker 24.0.5 2915 latest/stable canonical✓ -
k9s v0.27.4 155 latest/stable derailed -
microk8s v1.30.0 6783 1.30-strict/stable canonical✓ -
nvidia-assemble 3-36-gb8b0680 62 22/stable xnox -
nvidia-core22 535.161.08+mesa23.2.1 40 latest/stable canonical✓ -
pc 22-0.3 146 22/stable canonical✓ gadget
pc-kernel 5.15.0-107.117.1 1833 22/stable canonical✓ kernel
pciutils 3.3.1-3 3 latest/stable woodrow -
snapd 2.62 21465 latest/stable canonical✓ snapd
```

Output of `sudo snap logs nvidia-assemble`

```
024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + mknod -m 666 /dev/nvidiactl c 195 255
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + mknod -m 666 /dev/nvidia-modeset c 195 254
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[2226]: + sed -n s|^\([0-9]*\) nvidia-uvm$|\1|p /proc/devices
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + major=504
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + [ -n 504 ]
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + mknod -m 666 /dev/nvidia-uvm c 504 0
2024-05-18T05:37:17Z nvidia-assemble.nvidia-assemble[1767]: + mknod -m 666 /dev/nvidia-uvm-tools c 504 1
2024-05-18T05:37:17Z systemd[1]: snap.nvidia-assemble.nvidia-assemble.service: Deactivated successfully.
2024-05-18T05:37:17Z systemd[1]: Finished Service for snap application nvidia-assemble.nvidia-assemble.
2024-05-18T05:37:17Z systemd[1]: snap.nvidia-assemble.nvidia-assemble.service: Consumed 1.273s CPU time.
```

Error found when running `sudo snap logs docker`

```
failed to generate CDI spec: failed to create edits common for entities: failed to create discoverer for common entities: failed to create discoverer for driver files: failed to create discoverer for driver libraries: failed to get libraries for driver version: failed to locate libcuda.so.535.161.08: pattern libcuda.so.535.161.08 not found"
```

Output of `sudo pciutils.lspci -v | grep -i nvidia`

```
21:00.0 VGA compatible controller: NVIDIA Corporation Device 2204 (rev a1) (prog-if 00 [VGA controller])
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nvidia_drm, nvidia
21:00.1 Audio device: NVIDIA Corporation Device 1aef (rev a1)
```

Truncated output of `modinfo nvidia`

```
filename: /lib/modules/5.15.0-107-generic/kernel/nvidia-535srv/nvidia.ko
firmware: nvidia/535.161.08/gsp_tu10x.bin
firmware: nvidia/535.161.08/gsp_ga10x.bin
alias: char-major-195-*
version: 535.161.08
supported: external
license: NVIDIA
srcversion: 1C6DE25E8197E808964F4CF
alias: pci:v000010DEd*sv*sd*bc06sc80i00*
alias: pci:v000010DEd*sv*sd*bc03sc02i00*
alias: pci:v000010DEd*sv*sd*bc03sc00i00*
depends: drm
retpoline: Y
name: nvidia
vermagic: 5.15.0-107-generic SMP mod_unload modversions
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Docker snap logs and the `nvidia-core22` and `nvidia-assemble` installation details included in the report, especially the missing `libcuda.so.535.161.08` CDI error. Trace how the Docker snap discovers NVIDIA driver libraries on Ubuntu Core 22 and compare that with the assembled device files; done means Docker reports an NVIDIA runtime and the CDI generation error is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, linux, shell
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.