modelscope / modelscope/ms-swift

Qwen3-VL-4B gptq量化, 使用自定义数据, ZeroDivisionError: float division by zero

Open
#6,770 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Describe the bug
尝试1: 直接传入dataset-path
尝试2: 将dataset 注册到data-info.json中, 量化时使用dataset-name

File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 286, in quantize_model
QuantEngine(args).quantize()
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 45, in quantize
gptq_quantizer = self.gptq_model_quantize(v2=(args.quant_method == 'gptq_v2'))
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/swift/llm/export/quant.py", line 280, in gptq_model_quantize
gptq_quantizer.quantize_model(self.model, self.tokenizer)
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/optimum/gptq/quantizer.py", line 640, in quantize_model
quant_outputs = gptq[name].fasterquant(
File "/home/ubuntu/miniforge3/envs/gptq_3vl/lib/python3.10/site-packages/auto_gptq/quantization/gptq.py", line 171, in fasterquant
logger.info(f"avg loss: {torch.sum(Losses).item() / self.nsamples}")
ZeroDivisionError: float division by zero

Your hardware and system info

Image

Additional context
{
"dataset_name": "AC_2400_ms",
"dataset_path": "/home/ubuntu/Project/ms-swift/scripts/ds/AC_2400_ms.json",
"columns": {
"messages": "messages",
"images": "images"
},
"tags": ["GUI", "AC_2400"]
}

单条格式:
{
"images": "/data/datasets/android_control/task_5348/screenshots_1.jpg",
"messages": [
{
"role": "user",
"content": " Task: In the 1weather app, view the weekly history of Philadelphia to understand the future conditions of the weather. \n Past Actions: []."
},
{
"role": "assistant",
"content": "<next_step>Scroll left on the right side of the screen.</next_step>\n{'bbox': [0, 0, 1080, 2400], 'type': 'swipe:right'}"
}
]
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with swift/llm/export/quant.py, especially QuantEngine.quantize and gptq_model_quantize, then inspect the auto_gptq fasterquant call shown in the traceback. Reproduce with the AC_2400_ms dataset configuration and its messages/images format; done means quantization no longer reaches a zero-sample division and the custom dataset is processed successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.