intel / intel/auto-round

[Bug]: CPU quantization can't get proper model size

Open
#2,293 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

### Problem Description

Qwen3.8-27B-FP8/Qwen3.8-27B-fp-w8afp8/, total 83G
Quantized model8 size larger than BF16

### Reproduction Steps

auto-round \
--model_name Qwen/Qwen3.8-27B \
--scheme FP8_STATIC \
--fp_layers "visual,linear_attn,mtp,lm_head,embed_tokens" \
--iters 0 \
--device_map auto \
--format auto_round \
--output_dir ./Qwen3.8-27B-FP8

### Environment Information

_No response_

### Error Logs

```shell

```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by running the provided auto-round command for Qwen/Qwen3.8-27B with the FP8_STATIC scheme and inspect the generated model size in ./Qwen3.8-27B-FP8. Compare it with the BF16 model and trace how CPU quantization accounts for model size; done means the FP8 quantized model size is reported or produced correctly and the reproduction is covered by a regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.