NVIDIA/Model-Optimizer

View on GitHub

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Stars: 3.8k
Forks: 604
Open beginner issues: 1
Indexed issues: 98
Avg merge: 2d 6h
Merged PRs (30d): 138

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 19, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

98 open issues indexed

Puzzletron Progress 6/8 (calculating one block scores) takes 10 to 20 times more than in tutorial Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

NVIDIA/Model-Optimizer#1667 · Opened Jun 10, 2026
Does Model Optimizer support QAT for GLM-5 with FP8, INT4, and NVFP4 quantization? Open

question

NVIDIA/Model-Optimizer#1666 · 6 comments · 1 assignee · Opened Jun 10, 2026
Puzzletron README initial setup fails with number of issues Open

bug documentation triaged

NVIDIA/Model-Optimizer#1637 · 1 comment · 1 assignee · Opened Jun 5, 2026
[Bug] NVFP4 W4A4 export: per-block weight_scale contains NaN bytes from unclamped FP32 → FP8 E4M3 cast Open

auto-fixable bug torch.quantization triaged

NVIDIA/Model-Optimizer#1633 · 1 comment · 1 assignee · Opened Jun 5, 2026
INT4 Quantization Export Open

bug torch.quantization triaged

NVIDIA/Model-Optimizer#1615 · 2 comments · 1 assignee · Opened Jun 3, 2026
generate_context_logits returns truncated logits when KV prefix cache is enabled, causing silently wrong loglikelihood evaluation Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 55/100

NVIDIA/Model-Optimizer#1577 · 1 comment · Opened Jun 1, 2026
Why int4 quant use special data size setting ? Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

NVIDIA/Model-Optimizer#1539 · Opened May 24, 2026
Why introduce the design of an `output_quantizer` within quantization operators? Open

question

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA/Model-Optimizer#1533 · Opened May 22, 2026
NVFP4 ONNX export fails when rotate=True (FWHT trace + post-processor) Open

bug feature onnx.quantization torch.quantization

NVIDIA/Model-Optimizer#1424 · 1 assignee · Opened May 10, 2026
Add VLM components to _default_disabled_quantizer_cfg? Open

feature request

NVIDIA/Model-Optimizer#1396 · 3 comments · 1 assignee · Opened May 6, 2026
Do you have plans to support Qwen Image Edit? Open

feature request

NVIDIA/Model-Optimizer#1366 · 1 assignee · Opened Apr 29, 2026
[Feature Request] DeepSeek-V4-Flash & DeepSeek-V4-Pro NVFP4 checkpoint Open

feature request

NVIDIA/Model-Optimizer#1346 · 1 comment · 1 reaction · 2 assignees · Opened Apr 27, 2026
[Feature Request] Kimi-K2.6 NVFP4 checkpoint and corresponding EAGLE3 draft model for offline speculative decoding Open

feature request

NVIDIA/Model-Optimizer#1308 · 24 comments · 1 reaction · 2 assignees · Opened Apr 21, 2026
Attention-only LoRA fine-tuning on NVFP4-quantized 122B MoE — community implementation + preprint Open

question

NVIDIA/Model-Optimizer#1294 · 1 comment · 1 assignee · Opened Apr 19, 2026
What’s the Correct Way to Quantize Qwen3.5 (MoE/Dense) to NVFP4? Open

question

NVIDIA/Model-Optimizer#1255 · 4 comments · 1 assignee · Opened Apr 14, 2026
Pre-Quantized Checkpoints: Gemma 4 models Open

feature request

NVIDIA/Model-Optimizer#1237 · 2 comments · 1 assignee · Opened Apr 11, 2026
Add _QuantGemma4TextExperts plugin for fused 3D MoE expert quantization Open

feature request torch.quantization triaged

NVIDIA/Model-Optimizer#1173 · 1 comment · 1 reaction · 1 assignee · Opened Apr 3, 2026
ONNX models generated by llm_export.py are missing some input and output nodes Open

bug triaged

NVIDIA/Model-Optimizer#1147 · 1 comment · 1 assignee · Opened Mar 31, 2026
[Model] Qwen3TTS Open

feature request onnx.quantization torch.quantization triaged

NVIDIA/Model-Optimizer#1090 · 2 comments · 1 assignee · Opened Mar 21, 2026
Speculative Decoding Open

documentation question torch.speculative triaged

NVIDIA/Model-Optimizer#1066 · 1 comment · 1 assignee · Opened Mar 18, 2026
Bug for register_attention_for_kv_quant Open

bug torch.quantization triaged

NVIDIA/Model-Optimizer#1064 · 1 comment · 1 assignee · Opened Mar 18, 2026
Quantization output using different ONNX Execution Providers Open

onnx.quantization question triaged

NVIDIA/Model-Optimizer#1032 · 1 comment · 1 reaction · 1 assignee · Opened Mar 12, 2026
ERROR：train sparse model with FDSP2 Open

bug torch.sparsity

NVIDIA/Model-Optimizer#821 · 1 comment · 1 assignee · Opened Jan 28, 2026
Add support for Z-Image architecture in Diffusers quantization examples Open

feature request torch.quantization

NVIDIA/Model-Optimizer#797 · 2 comments · 3 reactions · 1 assignee · Opened Jan 19, 2026
How to convert BF16 safetensors Generative AI models into NVFP4 like Qwen 2512, Wan 2.2, LTX 2, FLUX SRPO or any other model Open

bug

NVIDIA/Model-Optimizer#775 · 1 reaction · 1 assignee · Opened Jan 13, 2026
How to convert FP16 or FP8 model files to NVFP4 model files Open

question

NVIDIA/Model-Optimizer#770 · 7 comments · 1 assignee · Opened Jan 13, 2026
Quantizing Qwen VL 30bA3 Open

investigating model support question torch.quantization

NVIDIA/Model-Optimizer#716 · 5 comments · 1 reaction · 1 assignee · Opened Dec 21, 2025
Support PTQ for Qwen3 Omni Open

feature request investigating model support torch.quantization

NVIDIA/Model-Optimizer#647 · 4 comments · 1 assignee · Opened Dec 4, 2025
qwen3_8b eagle3 model train and vllm ckpt export Open

bug investigating

NVIDIA/Model-Optimizer#628 · 1 comment · 1 assignee · Opened Dec 2, 2025
FP8 model per channel weight quantification model export Open

bug feature investigating onnx.quantization

NVIDIA/Model-Optimizer#621 · 1 comment · 1 assignee · Opened Dec 1, 2025
Does calibration batch size affect quantization scales significantly? (Entropy Calibration) Open

feature investigating onnx.quantization question

NVIDIA/Model-Optimizer#619 · 1 assignee · Opened Nov 29, 2025
[Help wanted] CNN QAT: How to correctly export ONNX and use DeviceModel for inference Open

investigating model support question torch.quantization

NVIDIA/Model-Optimizer#604 · 1 comment · 1 assignee · Opened Nov 24, 2025
failed to quant the Qwen3-VL-30B-A3B Open

bug torch.quantization

NVIDIA/Model-Optimizer#569 · 1 comment · 1 assignee · Opened Nov 17, 2025
How to provide calibration data for INT8 quantization with dynamic ONNX input shapes Open

feature investigating onnx.quantization question

NVIDIA/Model-Optimizer#565 · 1 comment · 1 assignee · Opened Nov 15, 2025
Failed to save quantized model if some tensors offload to CPU Open

bug export/deploy investigating model support

NVIDIA/Model-Optimizer#491 · 2 comments · 1 assignee · Opened Nov 3, 2025
Diffusers Example quantize.py: --calib-size help text is misleading (acts as number of samples/prompts, not “calibration steps”). Suggest clearer name/alias and ceil batching. Open

documentation feature request investigating torch.quantization

NVIDIA/Model-Optimizer#247 · 1 assignee · Opened Aug 8, 2025
Is there any plan to support SVDQuant deployment? Open

feature request investigating model support torch.quantization

NVIDIA/Model-Optimizer#238 · 2 comments · 1 assignee · Opened Jul 31, 2025
NotImplementedError happend, when I trained sd2.1 model with fp8 quantization. Open

bug investigating model support torch.quantization

NVIDIA/Model-Optimizer#226 · 1 comment · 1 assignee · Opened Jul 3, 2025
the error for quantize.py Open

bug investigating model support torch.quantization

NVIDIA/Model-Optimizer#212 · 1 comment · 1 assignee · Opened Jun 13, 2025
Explicit INT8 Quantization Fails to Fuse Concat-Conv Block Compared to Implicit Mode Open

bug investigating model support needs trt/trtllm triage

NVIDIA/Model-Optimizer#174 · 6 comments · 1 assignee · Opened Apr 9, 2025
trt-modelopt is not compatible with pytorch FSDP？ Open

feature feature request investigating torch.quantization

NVIDIA/Model-Optimizer#139 · 3 comments · 1 assignee · Opened Feb 21, 2025
How to sparse the specified model parameters Open

feature investigating question torch.quantization

NVIDIA/Model-Optimizer#130 · 1 assignee · Opened Feb 13, 2025
How to do sparsity and quantization-aware-training ? Open

feature feature request investigating torch.quantization

NVIDIA/Model-Optimizer#122 · 3 comments · 1 assignee · Opened Jan 23, 2025
All the modules are disabled and onnx exporation is failed Open

bug investigating model support torch.quantization

NVIDIA/Model-Optimizer#104 · 1 assignee · Opened Nov 10, 2024
Can we use multi GPU while exporting (diffusers ) onnx model? Open

export/deploy feature feature request investigating

NVIDIA/Model-Optimizer#96 · 5 comments · 1 assignee · Opened Oct 29, 2024
cache_diffusion Open

feature feature request investigating torch.quantization

NVIDIA/Model-Optimizer#95 · 1 assignee · Opened Oct 29, 2024
Bringing Back Effective Quantization: Using ModelOPT for YOLO and Similar Architectures Open

feature feature request investigating onnx.quantization

NVIDIA/Model-Optimizer#83 · 2 comments · 1 assignee · Opened Oct 9, 2024
Quant Flux-dev OOM on L20 Open

bug investigating model support torch.quantization

NVIDIA/Model-Optimizer#72 · 6 comments · 1 assignee · Opened Sep 13, 2024

NVIDIA/Model-Optimizer

Get new issues in your inbox