modelscope / modelscope/ms-swift
Qwen3-VL-4B gptq量化, 使用自定义数据, ZeroDivisionError: float division by zero
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the bug
尝试1: 直接传入dataset-path
尝试2: 将dataset 注册到data-info.json中, 量化时使用dataset-name
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 286, in quantize_model
QuantEngine(args).quantize()
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 45, in quantize
gptq_quantizer = self.gptq_model_quantize(v2=(args.quant_method == 'gptq_v2'))
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 280, in gptq_model_quantize
gptq_quantizer.quantize_model(self.model, self.tokenizer)
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/optimum/gptq/quantizer.py", line 640, in quantize_model
quant_outputs = gptq[name].fasterquant(
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/auto_gptq/quantization/gptq.py", line 171, in fasterquant
logger.info(f"avg loss: {torch.sum(Losses).item() / self.nsamples}")
ZeroDivisionError: float division by zero
Your hardware and system info
Additional context
{
"dataset_name": "AC_2400_ms",
"dataset_path": "/home/ubuntu/Project/ms-swift/scripts/ds/AC_2400_ms.json",
"columns": {
"messages": "messages",
"images": "images"
},
"tags": ["GUI", "AC_2400"]
}
单条格式:
{
"images": "/data/datasets/android_control/task_5348/screenshots_1.jpg",
"messages": [
{
"role": "user",
"content": " Task: In the 1weather app, view the weekly history of Philadelphia to understand the future conditions of the weather. \n Past Actions: []."
},
{
"role": "assistant",
"content": "<next_step>Scroll left on the right side of the screen.</next_step>\n{'bbox': [0, 0, 1080, 2400], 'type': 'swipe:right'}"
}
]
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with swift/llm/export/quant.py, especially QuantEngine.quantize and gptq_model_quantize, then inspect the auto_gptq fasterquant call shown in the traceback. Reproduce with the AC_2400_ms dataset configuration and its messages/images format; done means quantization no longer reaches a zero-sample division and the custom dataset is processed successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100