[Feature]: Load SGLang-format W4AFP8 checkpoints (quant_method: w4afp8, e.g. PhalaCloud GLM-5.x) natively

🚀 The feature, motivation and pitch

Hopper users of the DeepSeek-V3.2 / GLM-5 family increasingly get their weights as SGLang-format W4AFP8 checkpoints (INT4 group-128 AWQ experts + FP8 block-scaled dense/attention layers, config.json quantization_config: {"quant_method": "w4afp8", "group_size": 128, "modules_to_not_convert": [...]}), e.g. PhalaCloud/GLM-5.3-W4AFP8, PhalaCloud/GLM-5.2-W4AFP8, and Kimi-K3 W4AFP8 variants. They are the practical way to serve these 700B+ models with 1M context on 8x H200.

TensorRT-LLM already implements this exact tensor layout: WInt4AFP8FusedMoEMethod in W4A8_CUSTOM loading mode reads {E}.w1/w3/w2.weight (int4 pairs packed in int8, [N, K/2]), {E}.w1/w3/w2.weight_scale_inv ([N, K/128]) and {E}.w1/w2/w3.input_scale, which is byte-for-byte what SGLang's w4afp8 writer emits (we verified PhalaCloud/GLM-5.3-W4AFP8 against Barrrrry/DeepSeek-R1-W4AFP8: identical names, dtypes, shapes and nibble packing; SGLang's kernel is a port of the TRT-LLM CUTLASS W4A8 grouped GEMM). What is missing is only the metadata: load_hf_quant_config does not recognise quant_method: w4afp8, so the model is built unquantized and weight loading fails.

Pitch: map quant_method == "w4afp8" in load_hf_quant_config to the same result as a MIXED_PRECISION hf_quant_config.json + quant_cfg.json pair — W4A8_AWQ for model.layers.L.mlp.experts on every MoE layer (including MTP layers), FP8_BLOCK_SCALES for the attention/dense/shared-expert projections, and the DeepSeek default exclude_modules (*kv_b_proj* etc.) — so these checkpoints load with no user-side files. We have this working today with a small external generator script that writes the two JSON files into a directory of symlinks; the generated map for GLM-5.3 is 550 entries (layers 0..78: self_attn.{fused_a,q_b_proj,o_proj}, self_attn.indexer.wq_b, mlp.shared_experts.{gate_up_proj,down_proj}, dense mlp.{gate_up_proj,down_proj}, mlp.experts).

Alternatives

• Keep requiring users to author hf_quant_config.json + quant_cfg.json (works, but is undocumented, and the MIXED_PRECISION path currently also needs #18393 and a manual *kv_b_proj* exclusion).
• Re-quantize with examples/quantization/quantize_mixed_precision_moe.py (hours per model, and duplicates a checkpoint that already exists).

Additional context

Measured on 8x H200 NVL with GLM-5.3 W4AFP8 through the metadata route: loads and serves at TP8/EP8, attention-DP and PP2xTP4; synthetic throughput at PP2xTP4 matched or beat SGLang TP8 on the same box. Blockers found on the way are reported separately (indexer wk block scale dropped at load; MIXED_PRECISION kv_b_proj exclusion; KV cache manager V2 failures under long prefix-shared traffic).

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Start in load_hf_quant_config, then compare the existing MIXED_PRECISION metadata path with the W4AFP8 checkpoint names, shapes, scales, and exclusions described in the issue. The change is done when the listed SGLang-format checkpoints load natively without user-generated metadata files and the resulting configuration covers MoE, dense, attention, and MTP layers.