MoE quantization: investigate AWQ (autoawq) support for MoE experts
Open
@titaiwangms is already working on this.
Since Aug 3, 2026.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 315
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 16
Description
Summary
Investigate and, if feasible, extend Olive's AWQ pass (olive/passes/pytorch/autoawq.py) to support MoE fused-expert weights, matching the RTN pass's moe=True support landed in microsoft/Olive#2584.
Why this is a different shape of problem than GPTQ/RTN
Unlike rtn.py and gptq.py (both native Olive implementations built on olive/common/quant/), autoawq.py is a thin wrapper around the external autoawq PyPI package (AutoAWQForCausalLM.from_pretrained(...).quantize(...)). Olive does not control the internal weight-replacement/calibration logic — it only configures and invokes the library.
What needs investigating (before any design work)
- Does the currently-pinned
autoawqversion already support quantizing MoE architectures we care about (Mixtral, gpt-oss, Qwen3-MoE, DeepSeek, etc.)? If yes for some subset, what output format does it produce for expert weights, and does it match (or can it be mapped to) Olive'sQuantTensor/on-disk buffer convention (olive/common/quant/state_dict.py), or does it need its own separate output path? - If
autoawqdoes not support MoE at all, is upstream support planned/in-progress, or would this require vendoring/patching behavior — and is that worth pursuing givenautoawqis a third-party dependency we don't control? - Scope check: does "AWQ support MoE" mean literally the
autoawqwrapper, or would a native olive-side AWQ implementation (mirroring thegptq.pyvsautogptq.pysplit) be more appropriate long-term? Worth a quick decision before committing to either wrapper-config-passthrough or a bigger native reimplementation.
References
- Related PR: microsoft/Olive#2584 (RTN MoE support)
- Related issue: microsoft/Olive#2599 (GPTQ MoE support — similar goal, different constraints since GPTQ is native)
- Original issue: microsoft/Olive#2583
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.