microsoft / microsoft/foundry-local

[CLI] Foundry Server Fails to Load Execution Provider for Nvidia GPU

Open
#1,074 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.6k
Forks
369
Avg merge
2d 17h
Merged PRs (30d)
39

Description

## Problem
Foundry server fails to load execution provider for Nvidia GPU, which in turn makes GPU models unavailable.

## Diagnostics
nvidia-smi output:
```
Sat Sep 5 17:05:00 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 616.56 KMD Version: 616.56 CUDA UMD Version: 13.4 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4060 ... WDDM | 00000000:01:00.0 Off | N/A |
| N/A 56C P0 13W / 85W | 0MiB / 8188MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
```

foundry server logs output:
```
2026-09-05 17:06:17.281 -07:00 [INF] Configuring services.
2026-09-05 17:06:17.916 -07:00 [INF] Initializing EpDetector with 5 bootstrapper(s).
2026-09-05 17:06:17.916 -07:00 [INF] EP registration deferred. Call DownloadAndRegisterEpsAsync to begin.
2026-09-05 17:06:17.916 -07:00 [INF] EpDetector initialization complete.
2026-09-05 17:06:17.937 -07:00 [INF] Loaded cached model info for 48 models. SavedAt:09/05/2026 16:29:06
2026-09-05 17:06:17.937 -07:00 [INF] Created AzureModelCatalog. URI:https://ai.azure.com/api/eastus/ux/v1.0 Cache
directory:C:\Users\stara/.foundry/cache/models
2026-09-05 17:06:17.937 -07:00 [INF] Getting latest info from the Azure catalog and for locally cached models.
2026-09-05 17:06:17.937 -07:00 [INF] Getting info for locally cached models.
2026-09-05 17:06:17.937 -07:00 [INF] Runtime versions: onnxruntime=1.26.0, onnxruntime-genai=0.14.1
2026-09-05 17:06:17.937 -07:00 [INF] Found 0 locally cached models.
2026-09-05 17:06:17.937 -07:00 [INF] EPs are not fully registered. Call DownloadAndRegisterEpsAsync to register EPs.
2026-09-05 17:06:17.937 -07:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:CoreInitialize
Status:Success Direct:True Time:0ms
2026-09-05 17:06:17.937 -07:00 [INF] Device: CPU,EPs: CPUExecutionProvider
2026-09-05 17:06:17.937 -07:00 [INF] Fetching latest models from Azure Foundry catalog
2026-09-05 17:06:17.942 -07:00 [INF] ExecuteCommandWithCallbackManaged: download_and_register_eps
Data:{"Params":{"Names":"CUDAExecutionProvider,WebGpuExecutionProvider,OpenVINOExecutionProvider,NvTensorRTRTXExecutionP
rovider,NvTensorRTRTXExecutionProvider"}}
2026-09-05 17:06:17.942 -07:00 [INF] DownloadAndRegisterEpsAsync called
2026-09-05 17:06:17.942 -07:00 [WRN] Unknown EP bootstrapper name(s): . Call DiscoverEps() and retry with supported
names only. Available EPs: CUDAExecutionProvider, WebGpuExecutionProvider, OpenVINOExecutionProvider,
NvTensorRTRTXExecutionProvider, NvTensorRTRTXExecutionProvider.
2026-09-05 17:06:17.942 -07:00 [INF] [Telemetry] AppName:foundry UserAgent:FoundryLocalCore
Command:AutoRegisterCertifiedEps Status:Failure Direct:True Time:0ms
2026-09-05 17:06:17.942 -07:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop
Command:AutoRegisterCertifiedEps Status:Failure Direct:True Time:0ms
2026-09-05 17:06:17.953 -07:00 [INF] Now listening on: http://127.0.0.1:56703
2026-09-05 17:06:17.953 -07:00 [INF] Application started. Press Ctrl+C to shut down.
2026-09-05 17:06:17.953 -07:00 [INF] Hosting environment: Production
2026-09-05 17:06:17.953 -07:00 [INF] Content root path: C:\Program
Files\WindowsApps\Microsoft.FoundryLocal_0.10.3.0_x64__8wekyb3d8bbwe\
2026-09-05 17:06:17.953 -07:00 [INF] Web application started. Listening on: http://127.0.0.1:56703
2026-09-05 17:06:17.953 -07:00 [INF] [Telemetry] AppName:foundry UserAgent:CoreNativeInterop Command:CoreServiceStart
Status:Success Direct:True Time:10ms
2026-09-05 17:06:18.483 -07:00 [INF] Detected region from catalog header: 'westus2'
2026-09-05 17:06:19.412 -07:00 [INF] RegionFallback: "Catalog" sticky region set to 'westus2' after 1 attempt(s).
2026-09-05 17:06:19.415 -07:00 [INF] Got 48 models from Azure Foundry catalog.
2026-09-05 17:06:19.415 -07:00 [INF] Populated model info for 0 models.
2026-09-05 17:06:19.415 -07:00 [INF] Getting info for locally cached models.
```

## System
- **OS:** Microsoft Windows 10.0.26200
- **Architecture:** X64
- **CPU:** 13th Gen Intel(R) Core(TM) i7-13700HX (24 logical cores)
- **GPU:** Intel Intel(R) UHD Graphics (128.0 MB), NVIDIA NVIDIA GeForce RTX 4060 Laptop GPU (7.8 GB)
- **RAM:** 19.5 GB available / 31.7 GB total
- **Disk:** 814.2 GB free / 1.8 TB total (C:\)
- **.NET:** .NET 9.0.18

## Service
- **State:** Ready
- **Web URLs:** http://127.0.0.1:56703
- **CLI version:** 0.10.3
- **Foundry Local Core:** 1.0.0
- **ORT:** 1.26.0
- **ORT GenAI:** 0.0.0

## Models
- **Available variants:** 48
- **Loaded:** 0
- **Cached:** 0

## Connectivity
- **Local service:** Reachable
- **Model registry:** Reachable

## Warnings
- No warnings.

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the Foundry server and inspect the execution-provider registration flow around DiscoverEps and DownloadAndRegisterEpsAsync, using the logged download_and_register_eps command as the entry point. Done means the CUDA execution provider registers successfully on the reported Windows/NVIDIA setup and GPU models become available.

Written by the indexing model from the issue text.

Assessment

Tech stack
cli
Domain
ai, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.