modelscope / modelscope/ms-swift

用swift量化错误

Open
#7,255 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

使用 swift export + gptq 量化 Qwen3-Reranker-4B 时,在 optimum.gptq 里遇到 layer_inputs[j] IndexError,是否目前不支持对 reranker 做 GPTQ?
Traceback (most recent call last):
File "/home/ps/www/ms-swift/swift/cli/export.py", line 5, in
export_main()
File "/home/ps/www/ms-swift/swift/llm/export/export.py", line 53, in export_main
return SwiftExport(args).main()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/www/ms-swift/swift/llm/base.py", line 49, in main
result = self.run()
^^^^^^^^^^
File "/home/ps/www/ms-swift/swift/llm/export/export.py", line 30, in run
quantize_model(args)
File "/home/ps/www/ms-swift/swift/llm/export/quant.py", line 286, in quantize_model
QuantEngine(args).quantize()
File "/home/ps/www/ms-swift/swift/llm/export/quant.py", line 45, in quantize
gptq_quantizer = self.gptq_model_quantize(v2=(args.quant_method == 'gptq_v2'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/www/ms-swift/swift/llm/export/quant.py", line 280, in gptq_model_quantize
gptq_quantizer.quantize_model(self.model, self.tokenizer)
File "/home/ps/anaconda3/envs/ms-swift/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/anaconda3/envs/ms-swift/lib/python3.11/site-packages/optimum/gptq/quantizer.py", line 630, in quantize_model
layer_inputs[j] = nested_move_to(layer_inputs[j], block_device)

IndexError: list index out of range

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Qwen3-Reranker-4B Swift export with GPTQ, then trace the call through swift/llm/export/quant.py into optimum/gptq/quantizer.py where layer_inputs[j] raises IndexError. Check whether reranker models are supported by this path and verify completion of the export or a clear compatibility outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.