Feature Request / Question: Support for Qwen3-Embedding-8B (.q4nx) on XDNA2
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Hey team,
I'm looking to run local RAG workflows directly on the NPU (currently deploying on a Strix Halo APU) and would heavily prefer to use Qwen3-Embedding-8B to offload the embedding process.
I see that the core Qwen 3 and Qwen 3.5 architectures are already mapped and supported for text generation and vision models in the registry. However, I noticed there isn't an officially published .q4nx file for the Qwen3-Embedding-8B variant yet.
I have a couple of questions regarding the path forward:
Official Support: Are there plans to officially publish and maintain the .q4nx for this embedding model in the registry so it can be natively pulled via flm pull?
Custom Compilation: If I were to compile it myself, are there any known gotchas for this specific architecture? note its an embedding model
If there is a reference script or specific flm-build configuration you recommend for adapting the feature-extraction graph of Qwen 3, that would be incredibly helpful.
Thanks for the great work on the engine!
Johnny
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Qwen 3 registry mappings and the flm-build configuration or feature-extraction graph referenced in the issue; verify how embedding models differ from text-generation models on XDNA2. Check whether a Qwen3-Embedding-8B .q4nx artifact can be built and pulled via flm pull, and document the required configuration or support decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100