microsoft / microsoft/winml-cli
Add a config-level reproduction recipe for the Qwen3 genai bundle under examples/recipes/
@xieofxie is already working on this.
Since Jul 27, 2026.
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Summary
Add a recipe under examples/recipes/ that lets users reproduce the Qwen3 (Qwen/Qwen3-0.6B) onnxruntime-genai bundle build at the config level, mirroring the existing per-(model, task, precision) recipe convention.
Deferred from the genai-bundle PR (#1081) to keep that PR focused; capturing the design notes here so the follow-up can execute directly.
Background
winml build -m Qwen/Qwen3-0.6B --ep qnn --device npu|auto -o out/bundle emits a genai bundle. It is composed of 4 ONNX components plus a code-driven assembly step:
| Component | model_type | task | device | precision |
|---|---|---|---|---|
ctx.onnx (transformer prefill) |
qwen3_transformer_only (decoder_prefill sub-model, seq=64) |
text-generation | NPU/QNN | w8a16 |
iter.onnx (transformer decode) |
qwen3_transformer_only (decoder_gen sub-model, seq=1) |
text-generation | NPU/QNN | w8a16 |
embeddings.onnx |
qwen3_embeddings_only |
feature-extraction | CPU | fp32 |
lm_head.onnx |
qwen3_lm_head_only |
feature-extraction | CPU | w4a32 |
genai_config.json + tokenizer |
assembly (recipe assemble step) |
— | — | — |
The 4 components each have a WinMLBuildConfig (config-expressible). The final assembly (genai_config.json, QNN stage session options, ONNX passes) is driven by the registered GenaiBundleRecipe in src/winml/modelkit/models/hf/qwen3/genai.py — it is not a WinMLBuildConfig, so it cannot be captured as a recipe JSON.
Proposed deliverable
Create examples/recipes/Qwen_Qwen3-0.6B/ containing:
- Per-component
WinMLBuildConfigJSONs following the existing role-suffix convention (cf.microsoft_trocr-*/image-to-text_fp16_config_{decoder,encoder}.json):text-generation_w8a16_config_context.jsontext-generation_w8a16_config_iterator.jsonfeature-extraction_fp32_config_embeddings.jsonfeature-extraction_w4a32_config_lm_head.json
- A
README.mdthat maps components → bundle, gives the one-command bundle build (assembly), and shows how to rebuild/inspect a single component withwinml build -c <config>.json. - Add the Qwen3 rows to the top-level
examples/recipes/README.mdmodel table.
Also cross-link docs/samples/qwen3-genai-bundle.md.
Design notes / gotchas for the implementer
- Faithful generation: generate the component configs from code (e.g.
winml config --model-type <component> --task <t> --device <d> --ep <ep> --precision <p>) rather than hand-authoring — the transformer configs are large (~28 layers of past/present KV tensors). - Transformer
max_cache_lenoverride: the composite defaultsmax_cache_lentomax_position_embeddings(40960 for Qwen3-0.6B), but the bundle recipe pinsmax_cache_len=2048/prefill_seq_len=64(seeQWEN3_GENAI_BUNDLE_RECIPEingenai.py). To faithfully reproduce the default bundle, the two transformer configs must reflectmax_cache_len=2048(and prefill seq=64 / decode seq=1). Decide whether to bake 2048 in or document the override —winml configuses a single--shape-configfor both composite sub-models, so per-sub-model seq differentiation comes from the model classes'_default_seq_len(64/1) automatically. - lm_head precision detail: the
w4a32lm_head is finalized toblock_size=32, accuracy_level=4byQwen3LMHeadOnlyQuantFinalizer(int8 MLAS GEMM). The regression where this fell back to128/0was fixed in #1081 (commit preservingmodel_typethrough the weight-only precision policy). The recipe README is a good place to note this so users can verify their rebuiltlm_head.onnxmatches. - Assembly is not config-expressible: make the README explicit that the per-component configs reproduce the 4 ONNX graphs, while the bundle directory (
genai_config.json+ layout) is produced by the one-commandwinml build(orscripts/qwen3.py export).
Acceptance criteria
-
examples/recipes/Qwen_Qwen3-0.6B/with the 4 component configs + README. - Configs load via
winml build -cand are faithful to the default bundle (transformermax_cache_len=2048; lm_headw4a32/block_size=32). - Qwen3 rows added to
examples/recipes/README.md. - README documents component→bundle mapping, the one-command assembly, and per-component rebuild.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.