docker-compose.yaml GPU example: capabilities [gpu, utility] omits compute, so libcuda.so.1 is not injected
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 49.2k
- Forks
- 4.5k
- Avg merge
- 1d 3m
- Merged PRs (30d)
- 239
Description
LocalAI version:
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 (sha256:02e823f5f25ddef3f4edb94a8960e506daa4f71595d7c75b041d52b6eaaf425b, LocalAI bcf196d), docker-compose.yaml at the same commit.
Environment, CPU architecture, OS, and Version:
Docker Desktop 29.7.2 (Compose 5.5.1) on Windows 11, WSL2 kernel 6.18.33.2-microsoft-standard-WSL2, x86_64. NVIDIA RTX 5070 Ti, driver 616.92.
Describe the bug
The commented NVIDIA example in docker-compose.yaml recommends:
# environment:
# NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
# init: true
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu, utility]
With capabilities: [gpu, utility], the container gets only the utility driver libraries. docker inspect shows "Capabilities":[["gpu","utility"]], and /usr/lib/x86_64-linux-gnu contains libnvidia-ml.so.1 and libdxcore.so but no libcuda.so.1. nvidia-smi works inside the container, so the setup looks fine, but every CUDA backend fails to start with ImportError: libcuda.so.1: cannot open shared object file. In my setup, also setting NVIDIA_DRIVER_CAPABILITIES=compute,utility in environment: (as the example suggests) did not change this.
Changing it to capabilities: [gpu, compute, utility] fixes it. libcuda.so.1 then shows up in /usr/lib/x86_64-linux-gnu and in ldconfig -p, and CUDA backends load.
To Reproduce
- Enable the legacy
driver: nvidiaexample fromdocker-compose.yamlas written, with a CUDA image such asmaster-gpu-nvidia-cuda-13. docker compose up -ddocker exec <container> sh -c 'ldconfig -p | grep libcuda.so.1'returns nothing. Loading a model on a Python CUDA backend fails with thelibcuda.so.1error above.
Expected behavior
Following the compose example gives a container where CUDA backends work.
Logs
ERROR Failed to load model ... error=failed to load model with internal loader: grpc service not ready: backend process exited with code 1: ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
Additional context
Suggested fix: use capabilities: [gpu, compute, utility] in the examples, and mention that compute is what brings in libcuda. I only tested the legacy driver: nvidia variant, on Docker Desktop/WSL2. I did not test the CDI (nvidia.com/gpu) variant, which may treat capabilities differently. It's worth checking the docs pages that show the same snippet as well.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the commented NVIDIA example in docker-compose.yaml and search the documentation pages for the same capabilities snippet. Reproduce the legacy driver variant if possible, then update the examples so CUDA backends receive libcuda.so.1; verify the compose configuration includes compute alongside gpu and utility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose
- Domain
- devops, documentation
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 84/100