microsoft / microsoft/onnxruntime
[CUDA] Qwen3.6-35B-A3B Throughput Optimization
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
This is used to track the progress of Qwen3.6-35B-A3B Throughput Optimization.
Related PRs:
For Qwen 3.6:
* olive-recipes
[Add Qwen3.6-35B-A3B MoE VLM recipe (CUDA + CPU)](https://github.com/microsoft/olive-recipes/pull/492)
* onnxruntime-genai
[Fix CUDA QMoE INT4 export for Qwen3.5/3.6 MoE models](https://github.com/microsoft/onnxruntime-genai/pull/2209)
https://github.com/microsoft/onnxruntime-genai/pull/2218
* cuda op / kernels:
https://github.com/microsoft/onnxruntime/pull/28980
https://github.com/microsoft/onnxruntime/pull/28985
https://github.com/microsoft/onnxruntime/pull/28986 (Not needed if we have shared expert optimization like below)
https://github.com/microsoft/onnxruntime/pull/29028
https://github.com/microsoft/onnxruntime/pull/29038 (Need to extend to block quantization)
https://github.com/microsoft/onnxruntime/pull/29013
https://github.com/microsoft/onnxruntime/pull/29824
https://github.com/microsoft/onnxruntime/pull/29818
Related issues:
[AddExternalInitializers copies device (GPU) OrtValues per session instead of using them in place
](https://github.com/microsoft/onnxruntime/issues/29009)
Contributor guide
Research direction
Start by reviewing the related olive-recipes, onnxruntime-genai, and ONNX Runtime PRs listed in the issue, including the CUDA op and kernel changes. Trace how those changes address Qwen3.6-35B-A3B throughput and determine the remaining work needed across the linked efforts. Done means the tracked optimization work is complete, but the issue does not define a specific benchmark or acceptance target.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, machine-learning
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100