Docker Model Runner on Windows fails to provision com.docker.nv-gpu-info.exe, breaking llama.cpp CUDA backend after Docker Desktop 4.82.0

Docker Model Runner on Windows fails to provision com.docker.nv-gpu-info.exe, breaking llama.cpp CUDA backend after Docker Desktop 4.82.0

Description

Docker Model Runner on Windows fails to correctly provision the NVIDIA GPU detection helper:

com.docker.nv-gpu-info.exe

into:

%USERPROFILE%\.docker\bin\inference

The issue appears to have started after Docker Desktop 4.82.0 (233772).

Docker Desktop 4.82.0 works correctly.

The issue is reproducible on later Docker Desktop versions and is still present in:

Docker Desktop 4.88.1 (237512)

The helper executable is included with Docker Desktop itself:

C:\Program Files\Docker\Docker\resources\model-runner\bin\com.docker.nv-gpu-info.exe

but it is missing from:

%USERPROFILE%\.docker\bin\inference

where Docker Model Runner expects to execute it.

Environment

• Windows
• WSL2 backend
• Docker Desktop 4.88.1 (237512)
• Docker Model Runner enabled
• llama.cpp backend
• NVIDIA CUDA GPU
• Docker Desktop 4.82.0 (233772) was the last version where this worked correctly for me

Actual behavior

Running:

docker model status

produces:

Docker Model Runner is running

BACKEND    STATUS         DETAILS
llama.cpp  Error          failed to install llama.cpp: failed to check CUDA 11 capability: fork/exec C:\Users\<user>\.docker\bin\inference\com.docker.nv-gpu-info.exe: The system cannot find the file specified.
diffusers  Not Installed
vllm       Not Installed  only supported on Linux 

Checking the expected location:

Test-Path "$env:USERPROFILE\.docker\bin\inference\com.docker.nv-gpu-info.exe"

returns:

False

However, the helper exists in the Docker Desktop installation:

Test-Path "C:\Program Files\Docker\Docker\resources\model-runner\bin\com.docker.nv-gpu-info.exe"

returns:

True

Workaround

Copying only the missing helper manually:

Copy-Item `
  "C:\Program Files\Docker\Docker\resources\model-runner\bin\com.docker.nv-gpu-info.exe" `
  "$env:USERPROFILE\.docker\bin\inference\com.docker.nv-gpu-info.exe"

immediately allows Docker Model Runner to continue installation.

After the first manual copy:

docker model status

changes from the error above to:

llama.cpp  Installing  downloading llama.cpp b9879-cuda

Docker then downloads and installs the CUDA backend.

The resulting version file is:

{"tag":"b9879-cuda","digest":"sha256:57130b8af4f80d754ee93e8855e74a9bc6c13e07f99b9becab720cce2658199e"}

and the inference directory contains the newly installed CUDA runtime, including:

cublas64_12.dll
cublasLt64_12.dll
cudart64_12.dll
ggml-cuda.dll
com.docker.llama-server.exe
...

However, during this backend installation/reconciliation, com.docker.nv-gpu-info.exe appears not to be included in the newly provisioned runtime.

In my testing I had to copy com.docker.nv-gpu-info.exe again after the new backend had been installed.

After doing that, Docker Model Runner starts normally:

Docker Model Runner is running

BACKEND    STATUS         DETAILS
llama.cpp  Running        llama.cpp b9879-cuda (sha256:57130b8af4f80d754ee93e8855e74a9bc6c13e07f99b9becab720cce2658199e) 72874f5
diffusers  Not Installed
vllm       Not Installed  only supported on Linux

Inference also works correctly:

docker model list

MODEL NAME               PARAMETERS  QUANTIZATION   ARCHITECTURE    MODEL ID
gemma4                   7.52B       MOSTLY_Q4_K_M  gemma4          533eb3ccab62
nomic-embed-text-v2-moe  475.29M     MOSTLY_F16     nomic-bert-moe  3e19972dda57

and:

docker model run gemma4 "Say hello in one sentence."

returns:

Hello there!

Additional observations

Before the workaround, my existing inference runtime contained the CUDA libraries and llama.cpp executable, but not:

com.docker.nv-gpu-info.exe

The Docker Desktop bundled directory does contain the helper:

C:\Program Files\Docker\Docker\resources\model-runner\bin\com.docker.nv-gpu-info.exe

After manually providing the helper, Docker Model Runner successfully detects CUDA support and downloads the correct b9879-cuda backend.

This strongly suggests that the failure is not caused by the NVIDIA driver, CUDA itself, WSL2, or llama.cpp.

It appears to be a Windows Docker Model Runner provisioning/reconciliation issue where com.docker.nv-gpu-info.exe is required from:

%USERPROFILE%\.docker\bin\inference

but is not copied there as part of the llama.cpp CUDA backend setup.

There may also be a second provisioning issue where installation of a new llama.cpp CUDA backend recreates or updates the inference runtime without preserving/provisioning this helper, requiring it to be copied again.

Expected behavior

Docker Desktop should automatically provision:

com.docker.nv-gpu-info.exe

into:

%USERPROFILE%\.docker\bin\inference

before performing CUDA capability detection.

Installing or updating the llama.cpp CUDA backend should also preserve or restore this helper so that Model Runner remains functional across backend updates and Docker Desktop restarts.

No manual copying from:

C:\Program Files\Docker\Docker\resources\model-runner\bin

should be required.

Regression

For my setup:

Docker Desktop 4.82.0 (233772) - works
Docker Desktop > 4.82.0        - affected
Docker Desktop 4.88.1 (237512) - still affected

The issue is especially problematic after a clean Docker Desktop installation because there is no previously provisioned inference runtime to fall back to.

Start by reproducing the failure with docker model status on Windows and checking whether %USERPROFILE%\.docker\bin\inference\com.docker.nv-gpu-info.exe exists while the bundled copy is present under C:\Program Files\Docker\Docker\resources\model-runner\bin. Trace the provisioning and backend reconciliation paths for the llama.cpp CUDA runtime. Done means the helper is provisioned and retained across installation, updates, and restarts, with Model Runner reporting the backend as Running without manual copying.