Guide for local ONNX model conversion to run on Copilot+ PCs?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 603
- PR merge metrics
- No merged PRs in 30d
Description
Per the blog article
Fara-7B is available on (opens in new tab)Microsoft Foundry (opens in new tab)and (opens in new tab)Hugging Face(opens in new tab). We are also releasing the implementation of Fara-7B in Magentic-UI, so that users can try it in a contained environment through the inference code provided. Additionally, users can download the model for Copilot+ PCs powered by Windows 11 from the AI Toolkit in VSCode and run it all on-device, taking advantage of NPU hardware acceleration.
I've tried going through AI Toolkit and I see no way to download the model for Copilot+ PCs, and attempts at doing manual model conversion for NPUs using QNNExecutionProvider from AI Toolkit all fail.
Am I missing something? I'd love to see a step-by-step guide, since I don't see the model listed at all in the model searching capabilities in AI Toolkit either.
I really would love to test this model out locally and take advantage of the NPU.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Fara-7B inference code and the AI Toolkit model-search and QNNExecutionProvider paths mentioned in the issue. Verify how the model can be downloaded or converted for Copilot+ PCs, then document the required steps and a successful local run using NPU acceleration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100