NVIDIA / NVIDIA/Model-Optimizer

[Bug] Qwen3-VL-30B-A3B NVFP4 quantization fails: hidden_size AttributeError in QuantQwen3VLMoeTextExperts

Open
#1,720 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
3.8k
Forks
604
Avg merge
2d 8h
Merged PRs (30d)
142

Description

Environment

  • nvidia-modelopt: 0.44.0 (latest PyPI + tested with latest source from main)
  • TensorRT-Edge-LLM: 0.8.0
  • Platform: NVIDIA Jetson AGX Thor (SM110, JetPack 7.2, CUDA 13.2)
  • Model: Qwen/Qwen3-VL-30B-A3B-Instruct (HuggingFace)
  • Python: 3.12

Problem

NVFP4 quantization of Qwen3-VL-30B-A3B-Instruct fails with a shape mismatch
in QuantQwen3VLMoeTextExperts during the MoE expert forward pass.

Error chain

  1. AttributeError: hidden_size in QuantQwen3VLMoeTextExperts
  2. mat1/mat2 shape mismatch in MoE expert forward

The error originates in modelopt's handling of this model's fused
gate_up_proj / refactored expert dimensions which do not match the
expected shapes in the MoE quantization code.

Steps to reproduce

tensorrt-edgellm-quantize-llm \
  --model_dir ~/models/Qwen3-VL-30B-A3B-Instruct/ \
  --output_dir ~/models/qwen3-vl-30b-nvfp4/onnx/ \
  --quantization nvfp4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the provided tensorrt-edgellm-quantize-llm command with Qwen/Qwen3-VL-30B-A3B-Instruct and NVFP4. Inspect QuantQwen3VLMoeTextExperts and the modelopt handling of fused gate_up_proj and refactored expert dimensions. Done means quantization completes without the hidden_size AttributeError or MoE expert shape mismatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.