microsoft / microsoft/foundry-local

[Bug] Foundry does not expose any CUDA-enabled model variants.

Open
#1,109 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
2.6k
Forks
369
Avg merge
2d 17h
Merged PRs (30d)
39

Description

### Describe the issue

**VM to host Microsoft Foundry Local 0.10.3 using an NVIDIA Tesla T4 GPU for local model inference.**

Foundry installs and operates successfully. Models can be downloaded, loaded, and executed. The NVIDIA driver and CUDA runtime are functioning correctly, and Foundry successfully downloads and registers the CUDA Execution Provider.

However, Foundry does not expose any CUDA-enabled model variants. Running:

foundry model list --variants

only returns model variants using:

Image

**Foundry logs confirm that CUDAExecutionProvider is successfully installed and registered:**

`CUDAExecutionProvider installed and ready for use.
CUDAExecutionProvider succeeded.

WebGpuExecutionProvider installed and ready for use.
WebGpuExecutionProvider succeeded.

Available EPs:
CPUExecutionProvider
WebGpuExecutionProvider
CUDAExecutionProvider`

Additionally, foundry status reports:

Image

while Windows and NVIDIA tools correctly identify the Tesla T4.

Image

Device Manager:
Image

Verified Tesla T4 appears via PowerShell:
Get-CimInstance Win32_VideoController

Image

Verified CUDA EP files were downloaded under:

%USERPROFILE%\.foundry\ep\cuda-ep

including:

onnxruntime_providers_cuda.dll
onnxruntime-genai-cuda.dll
cudart64_12.dll
cublas64_12.dll
cudnn64_9.dll

Verified foundry.modelinfo.json contains no CUDA model variants.
Verified foundry model list --variants only returns CPU/WebGPU variants.

This VM is hosted in Azure and the NVIDIA driver is installed via Virtual Machine - Settings Extension + applicaions blade- NvidiaGpuDriverWindows
SKU NC4as T4 v3

Microsoft Foundry Local 0.10.3
Windows 11 Build 26200
NVIDIA Tesla T4
NVIDIA Driver 591.59
CUDA 13.1

**Foundry Server Logs**

`PS C:\Windows\system32> foundry server logs
── daemon ── foundrylocald-2026-09-15.log ──
2026-09-15T13:44:25.369Z [INF] Foundry.Daemon.Startup: Binding IPC server
2026-09-15T13:44:25.370Z [INF] Foundry.Daemon.Ipc.IpcServer: IPC server listening at
foundry-cli-S-1-5-21-1616321936-482565587-436032346-5739.
2026-09-15T13:44:25.370Z [INF] Foundry.Daemon.Startup: Starting heartbeat publisher (cadence=00:00:05)
2026-09-15T13:44:25.370Z [INF] Foundry.Daemon.Startup: Activating noise suppressor
2026-09-15T13:44:25.370Z [INF] Foundry.Daemon.Startup: Calling FoundryLocalManager.CreateAsync
2026-09-15T13:44:25.604Z [INF] Foundry.Core: Foundry.Local.Core initialized successfully: Foundry Local Core was
successfully initialized.
2026-09-15T13:44:25.604Z [INF] Foundry.Daemon.Startup: Downloading and registering EPs
2026-09-15T13:46:21.247Z [INF] Foundry.Daemon.Startup: Starting SDK web service
2026-09-15T13:46:21.262Z [INF] Foundry.Daemon.Startup: Writing daemon discovery file (1 URLs)
2026-09-15T13:46:21.265Z [INF] Foundry.Daemon.Startup: Daemon Ready

── sdk (native) ── foundry.core20260915.log ──
2026-09-15 13:44:25.604 +00:00 [INF] Device: CPU,EPs: CPUExecutionProvider
2026-09-15 13:44:25.604 +00:00 [INF] Fetching latest models from Azure Foundry catalog
2026-09-15 13:44:25.604 +00:00 [INF] Runtime versions: onnxruntime=1.26.0, onnxruntime-genai=0.14.1
2026-09-15 13:44:25.604 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:CoreInitialize
Status:Success Direct:True Time:0ms
2026-09-15 13:44:25.605 +00:00 [INF] ExecuteCommandWithCallbackManaged: download_and_register_eps
Data:{"Params":{"Names":"CUDAExecutionProvider,WebGpuExecutionProvider"}}
2026-09-15 13:44:25.605 +00:00 [INF] DownloadAndRegisterEpsAsync called
2026-09-15 13:44:25.605 +00:00 [INF] DownloadAndRegisterEpsAsync: processing 2 bootstrapper(s) (userAgent:
FoundryLocalCore)
2026-09-15 13:44:25.605 +00:00 [INF] DownloadAndRegisterEpsAsync [1/2]: starting CUDAExecutionProvider.
2026-09-15 13:44:26.184 +00:00 [INF] Detected region from catalog header: 'eastus'
2026-09-15 13:44:27.233 +00:00 [INF] RegionFallback: "Catalog" sticky region set to 'eastus' after 1 attempt(s).
2026-09-15 13:44:27.237 +00:00 [INF] Got 48 models from Azure Foundry catalog.
2026-09-15 13:44:27.242 +00:00 [INF] Populated model info for 0 models.
2026-09-15 13:44:27.242 +00:00 [INF] Getting info for locally cached models.
2026-09-15 13:46:20.002 +00:00 [INF] CUDA EP successfully installed and ready for use.
InstallPath=C:\Users\Admin_CHannaford_Dev/.foundry/ep\cuda-ep,
Library=C:\Users\Admin_CHannaford_Dev/.foundry/ep\cuda-ep\onnxruntime_providers_cuda.dll, Version=unknown
2026-09-15 13:46:20.003 +00:00 [INF] DownloadAndRegisterEpsAsync [1/2]: CUDAExecutionProvider succeeded.
2026-09-15 13:46:20.003 +00:00 [INF] DownloadAndRegisterEpsAsync [2/2]: starting WebGpuExecutionProvider.
2026-09-15 13:46:21.247 +00:00 [INF] WebGPU EP successfully installed and ready for use.
InstallPath=C:\Users\Admin_CHannaford_Dev/.foundry/ep\webgpu-ep,
Library=C:\Users\Admin_CHannaford_Dev/.foundry/ep\webgpu-ep\onnxruntime_providers_webgpu.dll, Version=unknown
2026-09-15 13:46:21.247 +00:00 [INF] DownloadAndRegisterEpsAsync [2/2]: WebGpuExecutionProvider succeeded.
2026-09-15 13:46:21.247 +00:00 [INF] EP registration complete: 2 bootstrapper(s) succeeded in 115632ms. Available EPs:
CPUExecutionProvider, WebGpuExecutionProvider
2026-09-15 13:46:21.247 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:FoundryLocalCore
Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:115632ms
2026-09-15 13:46:21.247 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:115633ms
2026-09-15 13:46:21.262 +00:00 [INF] Now listening on: http://127.0.0.1:55536
2026-09-15 13:46:21.262 +00:00 [INF] Application started. Press Ctrl+C to shut down.
2026-09-15 13:46:21.262 +00:00 [INF] Hosting environment: Production
2026-09-15 13:46:21.262 +00:00 [INF] Content root path: C:\Program
Files\WindowsApps\Microsoft.FoundryLocal_0.10.3.0_x64__8wekyb3d8bbwe\
2026-09-15 13:46:21.262 +00:00 [INF] Web application started. Listening on: http://127.0.0.1:55536
2026-09-15 13:46:21.262 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:CoreServiceStart
Status:Success Direct:True Time:14ms
2026-09-15 14:42:06.814 +00:00 [INF] Getting latest info from the Azure catalog and for locally cached models.
2026-09-15 14:42:06.814 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:42:06.814 +00:00 [INF] Found 0 locally cached models.
2026-09-15 14:42:06.814 +00:00 [INF] Device: CPU,EPs: CPUExecutionProvider
2026-09-15 14:42:06.814 +00:00 [INF] Device: GPU,EPs: WebGpuExecutionProvider
2026-09-15 14:42:06.814 +00:00 [INF] Fetching latest models from Azure Foundry catalog
2026-09-15 14:42:07.144 +00:00 [INF] Detected region from catalog header: 'eastus'
2026-09-15 14:42:08.583 +00:00 [INF] Got 84 models from Azure Foundry catalog.
2026-09-15 14:42:08.584 +00:00 [INF] Populated model info for 0 models.
2026-09-15 14:42:08.584 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:42:08.585 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:ModelList
Status:Success Direct:True Time:1771ms
2026-09-15 14:42:08.587 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:42:08.587 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:ListDownloadedModels Status:Success Direct:True Time:0ms
2026-09-15 14:42:08.589 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:ListLoadedModels
Status:Success Direct:True Time:0ms
2026-09-15 14:42:08.590 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:42:08.590 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:ListDownloadedModels Status:Success Direct:True Time:0ms
2026-09-15 14:46:52.418 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:46:52.418 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:ListDownloadedModels Status:Success Direct:True Time:0ms
2026-09-15 14:46:52.420 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:ListLoadedModels
Status:Success Direct:True Time:0ms
2026-09-15 14:46:52.420 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:46:52.420 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:ListDownloadedModels Status:Success Direct:True Time:0ms
2026-09-15 14:46:52.421 +00:00 [INF] Getting info for locally cached models.
2026-09-15 14:46:52.421 +00:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:ListDownloadedModels Status:Success Direct:True Time:0ms
PS C:\Windows\system32>`

### To reproduce

Foundry local reproduces whenever ran.

### Urgency

_No response_

### Platform and architecture

Windows X64

### OS Version

Windows 11 Build 26200

### Installation type

Released package/binary

### Foundry Local version

Microsoft Foundry Local 0.10.3

### API or surface area

CLI

### Hardware acceleration/backend

WebGPU/CUDA

### Backend/runtime version

CUDA 13.1

Contributor guide

Open the contributing guide

Research direction

Start with `foundry model list --variants` and inspect `foundry.modelinfo.json`, using the server logs to trace device and execution-provider detection. Compare the registered CUDA provider with the model variants returned by the catalog. Done means CUDA-enabled variants appear in the CLI output and can be used for inference on the Tesla T4.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.