NVIDIA / NVIDIA/Model-Optimizer
What’s the Correct Way to Quantize Qwen3.5 (MoE/Dense) to NVFP4?
Open
@Edwardf0t1 is already working on this.
Since Apr 16, 2026.
question
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 604
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 142
Description
Hi, I’m currently unable to find any up-to-date documentation or guidance on properly quantizing Qwen3.5 (MoE/Dense) to NVFP4.
This process was working previously around the time #897 was merged, but now I’m running into issues. I’ve tried both --qformat nvfp4_mlp_only and --qformat nvfp4_experts_only, but neither seems to apply the expected quantization—the exported weights are still roughly equivalent in size to BF16.
I’d really appreciate any guidance or pointers on what might have changed or what I might be missing. Thanks in advance!
@Edwardf0t1 @cjluo-nv
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.