intel / intel/auto-round

[Bug]: save pretrained exception with quantized lm-head in auto-scheme

Open
#2,188 2 comments 0 reactions 1 assignee Claimed by @lvliang-intel View on GitHub
bug
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

### Problem Description

Image

exception
AR_AUTO_SCHEME_NSAMPLES=128 AR_AUTO_SCHEME_SEQLEN=512 CUDA_VISIBLE_DEVICES=6 python3 -m auto_round --model /models/Qwen3.8-27B --options "W2A16G64,W3A16" --iters 1000 --nsamples 512 --lr 2e-3 --shared_layers "q_proj,k_proj,v_proj" "gate_proj,up_proj" "in_proj_qkv,in_proj_z" --format auto_round --avg_bits 2.8 --layer_config "{lm_head:{scheme:W8A16}}" --output_dir /data2/wenhuach --tasks mmlu,gsm8k,leaderboard_mmlu_pro --enable_torch_compile

pass
AR_AUTO_SCHEME_NSAMPLES=128 AR_AUTO_SCHEME_SEQLEN=512 CUDA_VISIBLE_DEVICES=5 python3 -m auto_round --model /models/Qwen3.8-27B --options "W2A16G64,W3A16" --iters 1000 --nsamples 512 --lr 2e-3 --shared_layers "q_proj,k_proj,v_proj" "gate_proj,up_proj" "in_proj_qkv,in_proj_z" --format auto_round --avg_bits 2.8 --output_dir /data2/wenhuach/no-lmhead --tasks mmlu,gsm8k,leaderboard_mmlu_pro --enable_torch_compile

### Reproduction Steps

~

### Environment Information

~

### Error Logs

```shell

```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.