modelscope / modelscope/ms-swift

用gptq量化lora微调合并后的qwen3-next-80b-A3b模型,报错KeyError: 'self_attn.q_proj'

Open
#6,459 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Describe the bug

Image

模型是lora微调后合并的qwen3-next-80b-A3b模型
Your hardware and system info
命令是
OMP_NUM_THREADS=14
CUDA_VISIBLE_DEVICES=2,3,4,5,6
swift export
--model ms-swift/megatron_output/Qwen3-Next-80B-A3B-Instruct/v34-20251103-115154-hf_lora_e1
--dataset 'data_0101_1022_265w_converted.json'
--quant_n_samples 256
--quant_batch_size 1
--max_length 2048
--quant_method gptq_v2
--quant_bits 8
--output_dir Qwen3-Next-80B-A3B-Instruct-lora-GPTQ-Int8

absl-py 2.3.1
accelerate 1.11.0
addict 2.4.0
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.13.1
aiosignal 1.4.0
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
annotated-doc 0.0.3
annotated-types 0.7.0
anyio 4.11.0
anykeystore 0.2
apex 0.1
astor 0.8.1
async-timeout 5.0.1
attrdict 2.0.1
attrs 25.4.0
autopep8 2.3.2
binpacking 1.5.2
blake3 1.0.8
Brotli 1.1.0
cachetools 6.2.1
cbor2 5.7.1
certifi 2025.10.5
cffi 2.0.0
charset-normalizer 3.4.4
click 8.2.1
cloudpickle 3.1.2
cmake 4.1.2
compressed-tensors 0.11.0
contourpy 1.3.2
cpm-kernels 1.0.11
crcmod 1.7
cryptacular 1.6.2
cryptography 46.0.3
cupy-cuda12x 13.6.0
cycler 0.12.1
dacite 1.9.2
datasets 3.6.0
defusedxml 0.7.1
depyf 0.19.0
Device-SMI 0.5.1
dill 0.3.8
diskcache 5.6.3
distro 1.9.0
dnspython 2.8.0
einops 0.8.1
email-validator 2.3.0
exceptiongroup 1.3.0
fastapi 0.120.1
fastapi-cli 0.0.14
fastapi-cloud-cli 0.3.1
fastrlock 0.8.3
ffmpy 0.6.4
filelock 3.20.0
flash_attn 2.8.1
fonttools 4.60.1
frozendict 2.4.6
frozenlist 1.8.0
fsspec 2025.3.0
future 1.0.0
gguf 0.17.1
GPTQModel 5.2.0
gradio 5.49.1
gradio_client 1.13.3
greenlet 3.2.4
groovy 0.1.2
grpcio 1.76.0
h11 0.16.0
hf_transfer 0.1.9
hf-xet 1.2.0
httpcore 1.0.9
httptools 0.7.1
httpx 0.28.1
huggingface-hub 0.36.0
hupper 1.12.1
idna 3.11
importlib_metadata 8.7.0
iniconfig 2.3.0
interegular 0.3.3
jieba 0.42.1
Jinja2 3.1.6
jiter 0.11.1
jmespath 0.10.0
joblib 1.5.2
json_repair 0.52.3
jsonschema 4.25.1
jsonschema-specifications 2025.9.1
kiwisolver 1.4.9
lark 1.2.2
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.11.3
LogBar 0.1.8
Markdown 3.9
markdown-it-py 4.0.0
MarkupSafe 3.0.3
matplotlib 3.10.7
maturin 1.9.6
mdurl 0.1.2
megatron-core 0.13.2
mistral_common 1.8.5
ml_dtypes 0.5.3
modelscope 1.31.0
mpmath 1.3.0
ms_swift 3.10.0.dev0
msgpack 1.1.2
msgspec 0.19.0
multidict 6.7.0
multiprocess 0.70.16
networkx 3.4.2
ninja 1.13.0
nltk 3.9.2
numba 0.61.2
numpy 2.2.6
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-nccl-cu12 2.27.3
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvshmem-cu12 3.3.20
nvidia-nvtx-cu12 12.8.90
oauthlib 3.3.1
onnx 1.19.1
onnx-ir 0.1.12
onnxscript 0.5.4
openai 2.6.1
openai-harmony 0.0.4
opencv-python-headless 4.12.0.88
optimum 2.0.0
orjson 3.11.4
oss2 2.19.1
outlines_core 0.2.11
packaging 25.0
pandas 2.3.3
partial-json-parser 0.2.1.1.post6
PasteDeploy 3.1.0
pbkdf2 1.3
peft 0.17.1
pillow 11.3.0
pip 25.3
plaster 1.1.2
plaster-pastedeploy 1.0.1
pluggy 1.6.0
prometheus_client 0.23.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.4.1
protobuf 6.33.0
psutil 7.1.2
py-cpuinfo 9.0.0
pyarrow 22.0.0
pybase64 1.4.2
pybind11 3.0.1
pycodestyle 2.14.0
pycountry 24.6.1
pycparser 2.23
pycryptodome 3.23.0
pydantic 2.11.10
pydantic_core 2.33.2
pydantic-extra-types 2.10.6
pydub 0.25.1
Pygments 2.19.2
pyparsing 3.2.5
PyPcre 0.2.4
pyramid 2.0.2
pyramid-mailer 0.15.1
pytest 8.4.2
python-dateutil 2.9.0.post0
python-dotenv 1.2.1
python-json-logger 4.0.0
python-multipart 0.0.20
python3-openid 3.2.0
pytz 2025.2
PyYAML 6.0.3
pyzmq 27.1.0
random_word 1.0.13
ray 2.51.1
referencing 0.37.0
regex 2025.10.23
repoze.sendmail 4.4.1
requests 2.32.5
requests-oauthlib 2.0.0
rich 14.2.0
rich-toolkit 0.15.1
rignore 0.7.4
rouge 1.0.1
rpds-py 0.28.0
ruff 0.14.2
safehttpx 0.1.7
safetensors 0.6.2
scipy 1.15.3
semantic-version 2.10.0
sentencepiece 0.2.0
sentry-sdk 2.43.0
setproctitle 1.3.7
setuptools 78.1.1
shellingham 1.5.4
simplejson 3.20.2
six 1.17.0
sniffio 1.3.1
some-package 0.1
sortedcontainers 2.4.0
soundfile 0.13.1
soxr 1.0.0
SQLAlchemy 2.0.44
starlette 0.48.0
sympy 1.14.0
tensorboard 2.20.0
tensorboard-data-server 0.7.2
threadpoolctl 3.6.0
tiktoken 0.11.0
tokenicer 0.0.5
tokenizers 0.22.1
tomli 2.3.0
tomlkit 0.13.3
torch 2.8.0
torchao 0.14.1
torchaudio 2.8.0
torchvision 0.23.0
tqdm 4.67.1
transaction 5.0
transformer_engine 2.8.0
transformer_engine_cu12 2.8.0
transformer_engine_torch 2.8.0
transformers 4.57.1
transformers-stream-generator 0.0.5
translationstring 1.4
triton 3.4.0
trl 0.23.1
typer 0.20.0
typing_extensions 4.15.0
typing-inspection 0.4.2
tzdata 2025.2
urllib3 2.5.0
uvicorn 0.38.0
uvloop 0.22.1
velruse 1.1.1
venusian 3.1.1
vllm 0.10.2
watchfiles 1.1.1
WebOb 1.8.9
websockets 15.0.1
Werkzeug 3.1.3
wheel 0.45.1
WTForms 3.2.1
wtforms-recaptcha 0.3.2
xformers 0.0.32.post1
xgrammar 0.1.23
xxhash 3.6.0
yarl 1.22.0
zipp 3.23.0
zope.deprecation 6.0
zope.interface 8.0.1
zope.sqlalchemy 4.0
zstandard 0.25.0
Additional context
是不支持量化qwen3-next-80b-A3b模型吗

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the swift export entry point and reproduce the command using the reported merged Qwen3-Next model and gptq_v2. Trace where the KeyError: 'self_attn.q_proj' occurs and compare the model's module names with the GPTQ export path. Done means establishing whether this model is supported and either fixing the export failure or documenting the unsupported configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.