microsoft / microsoft/fara

Guide for local ONNX model conversion to run on Copilot+ PCs?

Open
#15 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.2k
Forks
603
PR merge metrics
No merged PRs in 30d

Description

Per the blog article

Fara-7B is available on (opens in new tab)Microsoft Foundry (opens in new tab)and (opens in new tab)Hugging Face(opens in new tab). We are also releasing the implementation of Fara-7B in Magentic-UI, so that users can try it in a contained environment through the inference code provided. Additionally, users can download the model for Copilot+ PCs powered by Windows 11 from the AI Toolkit in VSCode and run it all on-device, taking advantage of NPU hardware acceleration.

I've tried going through AI Toolkit and I see no way to download the model for Copilot+ PCs, and attempts at doing manual model conversion for NPUs using QNNExecutionProvider from AI Toolkit all fail.

Am I missing something? I'd love to see a step-by-step guide, since I don't see the model listed at all in the model searching capabilities in AI Toolkit either.

I really would love to test this model out locally and take advantage of the NPU.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Fara-7B inference code and the AI Toolkit model-search and QNNExecutionProvider paths mentioned in the issue. Verify how the model can be downloaded or converted for Copilot+ PCs, then document the required steps and a successful local run using NPU acceleration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.