microsoft / microsoft/foundry-local

[cli] Model load issue when running NPU models

Open
#829 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.6k
Forks
369
Avg merge
2d 17h
Merged PRs (30d)
39

Description

**Version used:** 0.10.0+174be11ea7aeacd8d0d67b0ba1daebec615284b1

**Device details:**

```powershell
> foundry status
╭──────────────┬───────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Section │ Metric │ Value │
├──────────────┼───────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ System │ OS │ Microsoft Windows 10.0.26300 │
│ System │ Architecture │ Arm64 │
│ System │ CPU │ Snapdragon(R) X 12-core X1E80100 @ 3.40 GHz (12 logical cores) │
│ System │ GPU │ Qualcomm Incorporated Qualcomm(R) Adreno(TM) X1-85 GPU (—) │
│ System │ NPU │ Qualcomm Technologies, Inc. Snapdragon(R) X Elite - X1E80100 - Qualcomm(R) Hexagon(TM) NPU │
│ System │ RAM │ 8.5 GB available / 15.6 GB total │
│ System │ Disk │ 579.6 GB free / 951.6 GB total (C:\) │
│ System │ .NET │ .NET 9.0.16 │
│ Service │ State │ Not running │
│ Service │ CLI version │ 0.10.0 │
│ Connectivity │ Local service │ Not reachable │
│ Warnings │ Warning 1 │ Qualcomm Adreno GPU detected. If acceleration fails, try a CPU model variant or update the GPU driver. │
╰──────────────┴───────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
Initially it was not showing NPU models and dint register QNNEP, so i uninstalled and installed WinML msix as suggested here. https://github.com/microsoft/Foundry-Local/issues/797 and this helped.

Now when running the models its killing the server:

```powershell
> foundry server start
■ Starting Foundry Local server

WebGpuExecutionProvider ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
QNNExecutionProvider ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%

● success: Server ready (http://127.0.0.1:56549)

> foundry run qwen2.5-coder-7b-instruct-qnn-npu:1
● error: Daemon closed the connection while awaiting response to op 'model.load'.
Hint: Run 'foundry server status' to inspect, or 'foundry server logs' for details

> foundry server status
State Not running
PID 11220
Started 2026-06-21 23:56:31Z
Uptime 44s
Web URLs http://127.0.0.1:56549
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with `foundry server start` followed by `foundry run qwen2.5-coder-7b-instruct-qnn-npu:1` on the reported Snapdragon Windows device. Start by running `foundry server logs` after the daemon closes, and compare the server status before and after model.load. Done means the QNN NPU model loads without terminating the server.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai-infra-agents, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
46/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.