NVIDIA/Model-Optimizer

View on GitHub

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Stars: 3.8k
Forks: 604
Open beginner issues: 0
Indexed issues: 93
Avg merge: 2d 8h
Merged PRs (30d): 142

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 19, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

93 issues indexed so far

qdq_to_dq fails on MaxViT due to unsupported Transpose after DequantizeLinear Open

bug

NVIDIA/Model-Optimizer#2442 · 1 comment · 2 assignees · Opened Sep 15, 2026
calling mtq.quantize causes AttributeError: 'tuple' object has no attribute 'dim' Open

question

NVIDIA/Model-Optimizer#2424 · 1 comment · 1 assignee · Opened Sep 13, 2026
Only 1 sample from AutoCast calibration data set is truly used Open

bug

NVIDIA/Model-Optimizer#2337 · 1 comment · 1 assignee · Opened Sep 4, 2026
After applying NVFP4 quantization followed by compression, RealQuantLinear fails to locate the corresponding GEMM implementation during inference. Open

bug

NVIDIA/Model-Optimizer#2330 · 2 comments · 1 assignee · Opened Sep 4, 2026
ONNX PTQ: no NVFP4 path for convolutional models Open

NVIDIA/Model-Optimizer#2281 · 1 assignee · Opened Aug 28, 2026
hf_ptq.py: deprecated --auto_quantize_bits CLI flag silently no-ops instead of enabling AutoQuantize Open

NVIDIA/Model-Optimizer#2230 · 1 comment · 1 assignee · Opened Aug 22, 2026
[Feature Request] DeepSeek-V4-Flash-0731 NVFP4 checkpoint Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/Model-Optimizer#2220 · 2 comments · Opened Aug 20, 2026
LUT-B Support Open

question

NVIDIA/Model-Optimizer#2204 · 1 comment · 1 assignee · Opened Aug 17, 2026
[Bug] init_quantized_weights / --low_memory_mode exports numerically broken NVFP4 (quantizes meta tensors before weights load) Open

NVIDIA/Model-Optimizer#2160 · 1 assignee · Opened Aug 12, 2026
Can `auto_quant` calculate the score for `kv-cache` separately? Open

question

NVIDIA/Model-Optimizer#2158 · 1 assignee · Opened Aug 12, 2026
Support MiniMax-H3 Visual VAE quantization in ModelOpt Open

feature request

NVIDIA/Model-Optimizer#2137 · 1 assignee · Opened Aug 10, 2026
# [ONNX][Autotune] Integrated quantization does not preserve AutoTune Q/DQ placement on ViT Open

bug

NVIDIA/Model-Optimizer#2123 · 1 assignee · Opened Aug 10, 2026
[ONNX PTQ] Support for third-party custom ORT/TRT plugins in static calibration Open

feature request

NVIDIA/Model-Optimizer#2016 · 7 comments · 1 assignee · Opened Jul 24, 2026
Recommended FP8 recipe for Blackwell (B200)? Per-tensor `FP8_DEFAULT_CFG` underperforms BF16 at prefill; NVFP4 much faster. Also: exported FP8 checkpoint has no KV `q/k/v_scale` Open

bug

NVIDIA/Model-Optimizer#2015 · 3 comments · 1 assignee · Opened Jul 24, 2026
NVFP4 support for Qwen3_5MoeExperts (fused MoE): quantizer registration + export serializer Open

NVIDIA/Model-Optimizer#2011 · 8 comments · 1 assignee · Opened Jul 23, 2026
[QAD] Request for exact training configurations used in arXiv:2601.20088 Open

NVIDIA/Model-Optimizer#2002 · 5 comments · 2 assignees · Opened Jul 21, 2026
Public preset `quant_cfg` changed from dict to list in 0.44 (breaking, undocumented?) Open

NVIDIA/Model-Optimizer#2001 · 7 comments · 1 assignee · Opened Jul 21, 2026
Nvidia Modelopt structured 2:4 weight sparsity showing no speed improvement compared to dense model Open

bug

NVIDIA/Model-Optimizer#1974 · 5 comments · 1 assignee · Opened Jul 14, 2026
Quantizing error for Qwen3.6 35B A3B Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 52/100

NVIDIA/Model-Optimizer#1933 · 5 comments · Opened Jul 7, 2026
[Bugs] Second batch of verified findings from the unit-coverage initiative (fold_weight crash on Llama-4/GPT-OSS MoE, KD-loss default footgun, and smaller items) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/Model-Optimizer#1926 · 4 comments · Opened Jul 6, 2026
[Coverage] Direct unit-test suites for five untested core modules (conversion, core_utils, tensor_quantizer, model_calib, export/postprocess) — plus bugs found along the way Open

NVIDIA/Model-Optimizer#1902 · 5 comments · 1 assignee · Opened Jul 5, 2026
NVFP4 quantized (shared) lm_head fails fake-quant in training forward: ScaledE4M3Function 'Only support E=4 & M=3' Open

bug investigating torch.quantization

Difficulty 4/5 3-5 days Newbie friendliness 55/100

NVIDIA/Model-Optimizer#1875 · 4 comments · Opened Jul 1, 2026
Support for INT4_AWQ quantization for Nemotron-3-Nano-30B-A3B Open

bug fw-megatron torch.quantization

NVIDIA/Model-Optimizer#1865 · 3 comments · 1 assignee · Opened Jun 30, 2026
fsdp2_aware_weight_update hides the real error with an UnboundLocalError Open

bug investigating torch.quantization

NVIDIA/Model-Optimizer#1859 · 4 comments · 1 assignee · Opened Jun 30, 2026
Do not getting access to Nemotron-Pretraining-SFT-v1 (needed for a mintron distillation tutorial) Open

investigating question

NVIDIA/Model-Optimizer#1852 · 5 comments · 1 assignee · Opened Jun 29, 2026
[Puzzletron] Standalone export fails for heterogeneous distilled checkpoints Open

bug investigating torch.pruning

NVIDIA/Model-Optimizer#1848 · 4 comments · 1 assignee · Opened Jun 28, 2026
Nested Local Distillation for Puzzletron Open

feature request

NVIDIA/Model-Optimizer#1822 · 1 assignee · Opened Jun 25, 2026
Lazy Checkpoint Creation for Puzzletron Open

feature request

NVIDIA/Model-Optimizer#1821 · 1 assignee · Opened Jun 25, 2026
Improve Activation Scoring of Puzzletron Open

feature request

NVIDIA/Model-Optimizer#1820 · 1 assignee · Opened Jun 25, 2026
distill.py fails with AttributeError: DistillQwen3VLModel has no attribute 'output_layer' when using Qwen3.5-0.8B as student/teacher Open

feature request

NVIDIA/Model-Optimizer#1819 · 1 comment · 1 assignee · Opened Jun 25, 2026
Currently no support for pruning Nemotron Nano v3 with Puzzletron Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/Model-Optimizer#1814 · 1 comment · Opened Jun 24, 2026
Currently no support for pruning Qwen3.5 and Qwen3.6 with Puzzletron Open

feature request

NVIDIA/Model-Optimizer#1813 · 1 assignee · Opened Jun 24, 2026
Clarify recommended QAD configs for full-quality recovery in diffusers examples Open

question windows

NVIDIA/Model-Optimizer#1812 · 5 comments · 1 assignee · Opened Jun 24, 2026
Create a claude tutorial for adding a new model to puzzletron (qwen 3.5) Open

feature request torch.pruning

NVIDIA/Model-Optimizer#1780 · 1 comment · 1 assignee · Opened Jun 21, 2026
Puzzletron OOM during step 6 (one-block scoring) for Qwen 3.5-2B VLM Open

bug

NVIDIA/Model-Optimizer#1779 · 1 comment · 1 assignee · Opened Jun 21, 2026
When creating a claude skill for puzzletron, quantization tests fails slowing down development process Open

bug

Difficulty 5/5 Over a week Newbie friendliness 28/100

NVIDIA/Model-Optimizer#1772 · 1 comment · Opened Jun 19, 2026
puzzletron examples are in two places: examples/pruning/puzzletron and examples/puzzletron Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 48/100

NVIDIA/Model-Optimizer#1768 · Opened Jun 18, 2026
Create a claude skill for puzzletron compression algorithm Open

feature request

NVIDIA/Model-Optimizer#1762 · 1 comment · 1 assignee · Opened Jun 17, 2026
Test puzzletron tutorial step by step and collect feedback Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

NVIDIA/Model-Optimizer#1732 · 2 comments · Opened Jun 15, 2026
How to set the right quant cfg ? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/Model-Optimizer#1723 · Opened Jun 15, 2026
[Bug] Qwen3-VL-30B-A3B NVFP4 quantization fails: hidden_size AttributeError in QuantQwen3VLMoeTextExperts Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

NVIDIA/Model-Optimizer#1720 · Opened Jun 14, 2026
MiniMax M3 nvfp4 ckpt Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/Model-Optimizer#1708 · 1 comment · 2 reactions · Opened Jun 13, 2026
[RFC] NVIDIA Model Optimizer — Product Roadmap Open

roadmap

NVIDIA/Model-Optimizer#1699 · 4 comments · 1 assignee · Opened Jun 12, 2026
Add MBridge distillation for puzzletron Open

feature request

NVIDIA/Model-Optimizer#1679 · 1 comment · 1 assignee · Opened Jun 11, 2026
Restarting torchrun --nproc_per_node 2 examples/puzzletron/main.py fails Open

bug

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA/Model-Optimizer#1668 · 1 comment · Opened Jun 10, 2026
Puzzletron Progress 6/8 (calculating one block scores) takes 10 to 20 times more than in tutorial Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

NVIDIA/Model-Optimizer#1667 · Opened Jun 10, 2026
Does Model Optimizer support QAT for GLM-5 with FP8, INT4, and NVFP4 quantization? Open

question

NVIDIA/Model-Optimizer#1666 · 6 comments · 1 assignee · Opened Jun 10, 2026
Puzzletron README initial setup fails with number of issues Open

bug documentation triaged

NVIDIA/Model-Optimizer#1637 · 1 comment · 1 assignee · Opened Jun 5, 2026
[Bug] NVFP4 W4A4 export: per-block weight_scale contains NaN bytes from unclamped FP32 → FP8 E4M3 cast Open

auto-fixable bug torch.quantization triaged

NVIDIA/Model-Optimizer#1633 · 1 comment · 1 assignee · Opened Jun 5, 2026
INT4 Quantization Export Open

bug torch.quantization triaged

NVIDIA/Model-Optimizer#1615 · 2 comments · 1 assignee · Opened Jun 3, 2026

NVIDIA/Model-Optimizer

Get new issues in your inbox