[Bug]: CPU quantization can't get proper model size
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 175
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 99
Description
### Problem Description
Qwen3.8-27B-FP8/Qwen3.8-27B-fp-w8afp8/, total 83G
Quantized model8 size larger than BF16
### Reproduction Steps
auto-round \
--model_name Qwen/Qwen3.8-27B \
--scheme FP8_STATIC \
--fp_layers "visual,linear_attn,mtp,lm_head,embed_tokens" \
--iters 0 \
--device_map auto \
--format auto_round \
--output_dir ./Qwen3.8-27B-FP8
### Environment Information
_No response_
### Error Logs
```shell
```
### Additional Context
_No response_
Contributor guide
Research direction
Start by running the provided auto-round command for Qwen/Qwen3.8-27B with the FP8_STATIC scheme and inspect the generated model size in ./Qwen3.8-27B-FP8. Compare it with the BF16 model and trace how CPU quantization accounts for model size; done means the FP8 quantized model size is reported or produced correctly and the reproduction is covered by a regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100