[Bug]: Kimi-K2.6_mxfp4_fp8attn_LLMC vllm eval qkv scale is not loaded
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 175
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 99
Description
### Problem Description
Kimi-K2.6_mxfp4_fp8attn_LLMC quantized model run vllm eval can't load qkv scale
https://inteltf-jenk.sh.intel.com/job/Intel-GPU-key-model-test/1152/artifact/evaluation_Kimi-K2.6_mxfp4.log
### Reproduction Steps
cd /software/data/jenkins/CS09_B200/workspace/Intel-GPU-key-model-test/neural-compressor/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/kimi
bash run_benchmark.sh --model_path=/home/hshen/suyue/saved_models/Kimi-K2.6_mxfp4_fp8attn_LLMC/Kimi-K2.6-mxfp-w4g32 --tasks=hellaswag,piqa,mmlu,gsm8k --static_attention_dtype=fp8
### Environment Information
https://inteltf-jenk.sh.intel.com/job/Intel-GPU-key-model-test/1152/artifact/evaluation_Kimi-K2.6_mxfp4.log
### Error Logs
```shell
(Worker_TP0 pid=7609) WARNING 09-11 08:05:24 [weight_utils.py:1462] Found k_scale in the checkpoint (e.g. layers.3.self_attn.attn.k_scale), but not found the expected name in the model (e.g. layers.3.self_attn.attn.attn.k_scale). k_scale is not loaded.
(Worker_TP0 pid=7609) WARNING 09-11 08:05:24 [weight_utils.py:1462] Found q_scale in the checkpoint (e.g. layers.3.self_attn.attn.q_scale), but not found the expected name in the model (e.g. layers.3.self_attn.attn.attn.q_scale). q_scale is not loaded.
(Worker_TP0 pid=7609) WARNING 09-11 08:05:24 [weight_utils.py:1462] Found v_scale in the checkpoint (e.g. layers.3.self_attn.attn.v_scale), but not found the expected name in the model (e.g. layers.3.self_attn.attn.attn.v_scale). v_scale is not loaded.
```
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.