[Bug]: MIXED_PRECISION quant config lacks the default kv_b_proj exclusion; GLM-5 (192/256 head dims) crashes in load_kv_b_proj_and_k_b_proj_trans

System Info

• GPU: 8x NVIDIA H200 NVL (SM90; two 4-GPU NVLink islands, cross-island over PCIe/UPI), driver 595.71.05
• Host: Ubuntu 24.04.4, 2 TiB RAM, Xeon 6747P
• Container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc25 (CUDA 13.2, Python 3.12.3, torch 2.12.0a0+nv26.5, transformers 5.5.4)
• TensorRT-LLM: 1.3.0rc25 (tag v1.3.0rc25, 785c948); the code paths cited below were also checked on main @ a6616d6f (2026-09-03)
• Backend: PyTorch (trtllm-serve --backend pytorch)

Who can help?

@NVShreyas (GLM-5 support, #11990) @yizhang-nv (#18374)

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

GLM-5 attention heads are qk_nope_head_dim=192 + v_head_dim=256, so the FP8 block-scaled kv_b_proj (F8_E4M3 [28672, 512], weight_scale_inv F32 [224, 4]) has 3.5 scale rows per head and the scale tensor cannot be split per head. The plain-FP8 config path knows this and always adds ["*kv_b_proj*", "*k_b_proj*", "*eh_proj"] to exclude_modules (tensorrt_llm/_torch/model_config.py, load_hf_quant_config, comment "kv_b_proj must always be excluded: FP8 128x128 block boundaries don't necessarily align with per-head dim boundaries (e.g. GLM-5 has qk_nope_head_dim=192)"), which routes the loader to load_kv_b_proj_and_k_b_proj_trans_dequant.

The MIXED_PRECISION path (hf_quant_config.json with quant_algo: MIXED_PRECISION + per-layer quant_cfg.json, i.e. the format of Barrrrry/DeepSeek-R1-W4AFP8 and of W4AFP8 GLM checkpoints such as PhalaCloud/GLM-5.3-W4AFP8) only takes exclude_modules from the JSON and never adds that default. The loader's decision is dequant_kv_b_proj = quant_config.is_module_excluded_from_quantization("kv_b_proj") (modeling_deepseekv3.py, DeepseekV3WeightLoader.load_weights), so for a GLM checkpoint it takes the block-scale split path and crashes:

File ".../tensorrt_llm/_torch/models/modeling_deepseekv3.py", line 419, in load_weights
    kv_b_proj_scale, k_b_proj_trans_scale = load_kv_b_proj_and_k_b_proj_trans(name, is_scale=True)
File ".../tensorrt_llm/_torch/models/modeling_deepseekv3.py", line 241, in load_kv_b_proj_and_k_b_proj_trans
    kv_b_proj = weights[f"{module_name}.{weight_name}"][:].unflatten(
RuntimeError: unflatten: Provided sizes [64, 3] don't multiply up to the size of dim 0 (224) in the input tensor

Steps: take PhalaCloud/GLM-5.3-W4AFP8 (SGLang W4AFP8 layout, identical tensor layout to Barrrrry/DeepSeek-R1-W4AFP8), add hf_quant_config.json = {"quantization": {"quant_algo": "MIXED_PRECISION", "kv_cache_quant_algo": null}} and a quant_cfg.json listing model.layers.L.self_attn.{fused_a,q_b_proj,kv_b_proj,o_proj} etc. as FP8_BLOCK_SCALES and model.layers.L.mlp.experts as W4A8_AWQ (with #18393 applied, otherwise the MIXED_PRECISION assertion fires first), then trtllm-serve <dir> --tp_size 8 --ep_size 8.

Expected behavior

kv_b_proj is dequantized to BF16 for head dims that are not 128-aligned regardless of which quant-config path produced the model config, as the plain-FP8 path does.

actual behavior

Weight loading aborts with the unflatten error above. Adding "exclude_modules": ["*kv_b_proj*", "*k_b_proj*", "*eh_proj"] to hf_quant_config.json works around it (that is how we run today).

additional notes

• Fix proposed in a PR (linked below): apply the same default exclusion in ModelConfig._build_modelopt_quant_config when the MIXED_PRECISION per-layer map contains an FP8 block-scaled self_attn.kv_b_proj entry, so the module is built unquantized and the loader takes the dequant path exactly as on the FP8 config path. A loader-only decision is not enough: the module must already be unquantized for the dequantized weight to be copied in.
• Only *kv_b_proj* is load-bearing on the PyTorch flow; *k_b_proj* and *eh_proj are inert there.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Start in tensorrt_llm/_torch/model_config.py at load_hf_quant_config and _build_modelopt_quant_config, then trace DeepseekV3WeightLoader.load_weights in tensorrt_llm/_torch/models/modeling_deepseekv3.py. Reproduce with the cited GLM-5.3-W4AFP8 configuration and trtllm-serve command. Done means MIXED_PRECISION applies the required exclusion and GLM loading avoids the unflatten error through the dequantization path.