microsoft / microsoft/Olive

MoE quantization: investigate AWQ (autoawq) support for MoE experts

Open
#2,600 0 comments 0 reactions 1 assignee View on GitHub

@titaiwangms is already working on this.

Since Aug 3, 2026.

Dominant language
Python
Stars
2.4k
Forks
315
Avg merge
4d 11h
Merged PRs (30d)
16

Description

Summary

Investigate and, if feasible, extend Olive's AWQ pass (olive/passes/pytorch/autoawq.py) to support MoE fused-expert weights, matching the RTN pass's moe=True support landed in microsoft/Olive#2584.

Why this is a different shape of problem than GPTQ/RTN

Unlike rtn.py and gptq.py (both native Olive implementations built on olive/common/quant/), autoawq.py is a thin wrapper around the external autoawq PyPI package (AutoAWQForCausalLM.from_pretrained(...).quantize(...)). Olive does not control the internal weight-replacement/calibration logic — it only configures and invokes the library.

What needs investigating (before any design work)

  1. Does the currently-pinned autoawq version already support quantizing MoE architectures we care about (Mixtral, gpt-oss, Qwen3-MoE, DeepSeek, etc.)? If yes for some subset, what output format does it produce for expert weights, and does it match (or can it be mapped to) Olive's QuantTensor/on-disk buffer convention (olive/common/quant/state_dict.py), or does it need its own separate output path?
  2. If autoawq does not support MoE at all, is upstream support planned/in-progress, or would this require vendoring/patching behavior — and is that worth pursuing given autoawq is a third-party dependency we don't control?
  3. Scope check: does "AWQ support MoE" mean literally the autoawq wrapper, or would a native olive-side AWQ implementation (mirroring the gptq.py vs autogptq.py split) be more appropriate long-term? Worth a quick decision before committing to either wrapper-config-passthrough or a bigger native reimplementation.

References

  • Related PR: microsoft/Olive#2584 (RTN MoE support)
  • Related issue: microsoft/Olive#2599 (GPTQ MoE support — similar goal, different constraints since GPTQ is native)
  • Original issue: microsoft/Olive#2583

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.