[DeepSeek-V4] Loading nvidia/DeepSeek-V4-Pro-NVFP4 crashes: 'Linear' object has no attribute 'weight_scale' in fused_a loader (MIXED_PRECISION attn excluded → built BF16 but stored as FP8)

System / environment

• Images (both reproduce, empirically verified 2026-07-09): nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc15.post1 and 1.3.0rc20 (NVIDIA Release 26.02, PyTorch 2.11.0a0). Note the DeepSeek-V4 model file was refactored between the two: in rc15.post1 the loader lives in _torch/models/modeling_deepseekv4.py; in rc20 it was folded into _torch/models/modeling_deepseekv3.py (there is no modeling_deepseekv4.py in rc20). The bug survives the refactor — see line refs for both below.
• Hardware: single node, 8× B300. Failure is during weight loading (per-rank, at Loading weights 0%), so it is hardware/parallelism independent — reproduces with TP4 and reproduces identically under both monolithic trtllm-serve serve and PD-disaggregated serving.
• Checkpoint: nvidia/DeepSeek-V4-Pro-NVFP4 — the ModelOpt experts-only NVFP4 re-quantization (hf_quant_config.json → producer.version: "dsv4-nvfp4-experts").

Reproduce

trtllm-serve serve /path/to/DeepSeek-V4-Pro-NVFP4 \
  --tp_size 4 --host 0.0.0.0 --port 8001
# crashes before serving, during model weight loading — no tokenizer/parser flags needed
# to reproduce. (In rc20 the `--tool_parser deepseek_v4` value was also removed; the valid
# set is deepseek_v3/deepseek_v31/deepseek_v32 — orthogonal to this bug.)

Actual behavior

File ".../tensorrt_llm/_torch/models/modeling_deepseekv4.py", line 1030, in load_weights
    _copy_deepseek_v4_fused_a_weight_scale(module, fused_a, fused_a_scale)
File ".../tensorrt_llm/_torch/models/modeling_deepseekv4.py", line 240, in _copy_deepseek_v4_fused_a_weight_scale
    if tuple(module.weight_scale.shape) == tuple(fused_a_scale.shape):
File ".../torch/nn/modules/module.py", line 1967, in __getattr__
    raise AttributeError(...)
AttributeError: 'Linear' object has no attribute 'weight_scale'

Root cause (traced through the code)

This checkpoint is MIXED_PRECISION: only the routed MoE experts are NVFP4; the attention projections are stored as native DeepSeek block-scale FP8 — every layers.N.attn.{wq_a,wkv,wo_a,wo_b,wq_b} leaf carries both .weight (fp8) and .scale. Its hf_quant_config.json:

{
  "producer": {"name": "modelopt", "version": "dsv4-nvfp4-experts"},
  "quantization": {
    "quant_algo": "MIXED_PRECISION",
    "quantized_layers": { "layers.0.ffn.experts": {"quant_algo": "NVFP4", "group_size": 16}, "...": "... (61 layers)" },
    "exclude_modules": ["*.attn.*", "*.ffn.shared_experts.*", "head", "mtp.*"]
  }
}

1. The attention Linears (q_a_proj / kv_a_proj_with_mqa, and MTP e_proj/h_proj) are built with model_config.get_quant_config() called with no name (e.g. modeling_deepseekv4.py:2177/2185/2197/2208). ModelConfig.get_quant_config(name=None) returns the global QuantConfig (model_config.py:249-250).
2. The per-layer table (quantized_layers → per_layer_quant_configs) is consulted only when a name is passed, and raises ValueError on a miss (model_config.py:252-254). In practice it is used only for the MoE experts (which are wired through a separate override_quant_config, modeling_deepseekv4.py:1467/1504). So no hf_quant_config.json / quant_cfg.json entry can reach the attention modules.
3. The global algo is MIXED_PRECISION + exclude_modules: "*.attn.*" (matched via fnmatch, quantization/quantize.py:51) → the attn Linear is built unquantized (BF16), with no weight_scale.
4. In load_weights, nvfp4_fused_a is False (attn weights are FP8, not FP4), so the code takes the else branch (~modeling_deepseekv4.py:1015-1032). That branch finds kv_a_proj_with_mqa.weight_scale_inv in the checkpoint and calls _copy_deepseek_v4_fused_a_weight_scale(module, fused_a, fused_a_scale), which dereferences module.weight_scale → AttributeError.

Note the same branch also does module.weight.data.copy_(fused_a) where fused_a is the FP8 weight and module.weight is BF16 — i.e. even if the weight_scale access were guarded, the FP8 weight would be loaded into a BF16 module without applying the block scale, silently producing wrong values. The branch implicitly assumes the fused_a module is FP8_BLOCK_SCALES.

Expectation / suggested fix

When the fused_a target module is unquantized (excluded → BF16) but the checkpoint stores it as block-scale FP8, the else branch should dequantize (weight_dequant, already present at modeling_deepseekv4.py:97-147) the FP8 fused_a with its fused_a_scale and copy the resulting BF16 into module.weight, instead of copying the scale onto a non-existent module.weight_scale. (Equivalently: guard on hasattr(module, "weight_scale") and branch to dequant.)

What we ruled out

• Image version — reproduced on both 1.3.0rc15.post1 and 1.3.0rc20 (empirically). rc20 refactored the V4 loader into modeling_deepseekv3.py and reworked model_config.py, but still sets exclude_modules from the HF quant config and crashes identically (rc20 crash site modeling_deepseekv3.py:~557).
• PD-disaggregated vs monolithic — identical crash (both workers run trtllm-serve serve; loading is the same code path).
• Removing *.attn.* from exclude_modules — still crashes; under a MIXED_PRECISION global there is no per-module algo for the now-un-excluded attn, so it is not built as FP8.
• Missing quant_cfg.json — neither the checkpoint nor the ModelOpt extended file exists in the repo; adding per-layer entries there cannot help, since attn queries get_quant_config() without a name (point 2 above).

Additional note

The native DeepSeek checkpoint deepseek-ai/DeepSeek-V4-Pro (FP8 + MXFP4 routed experts; config-level quantization_config: {quant_method: fp8, weight_block_size: [128,128]}, no exclude_modules) loads and serves fine — verified end-to-end on rc15.post1, TP4 on 4× B300: weights load to 100%, Application startup complete, and chat completions generate correctly. Because the global algo is FP8_BLOCK_SCALES (not MIXED_PRECISION) and attn is not excluded, every attn Linear is built as FP8_BLOCK_SCALES and has a weight_scale. So TensorRT-LLM can serve DeepSeek-V4-Pro on this hardware; the crash is specific to the ModelOpt experts-only NVFP4 packaging (nvidia/DeepSeek-V4-Pro-NVFP4), where attn is exclude_modules'd (→ built BF16) yet physically stored as FP8 with a scale.

Start with load_weights and _copy_deepseek_v4_fused_a_weight_scale in _torch/models/modeling_deepseekv4.py for rc15.post1, then compare the corresponding path in modeling_deepseekv3.py for rc20. Read the existing weight_dequant helper and verify that loading nvidia/DeepSeek-V4-Pro-NVFP4 handles excluded BF16 attention modules without crashing or silently copying scaled FP8 values incorrectly.