modelscope / modelscope/ms-swift
Qwen3Omni lora微调后推理报错
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
使用qwen3-omni的megaton微调后,进行推理出现报错:
ValueError: Qwen3OmniMoeThinkerForConditionalGeneration does not support LoRA yet.
推理脚本:
CUDA_VISIBLE_DEVICES=0,1 \
MAX_PIXELS=1003520 \
swift infer \
--model Qwen3-Omni/Qwen3-Omni-30B-A3B-Instruct \
--adapters ckpt/v4-20251212-163234/checkpoint-3 \
--merge_lora false \
--stream true \
--infer_backend vllm \
--val_dataset ms-swift/data/train_test.jsonl \
--vllm_gpu_memory_utilization 0.9 \
--vllm_tensor_parallel_size 2 \
--vllm_max_model_len 32768 \
--max_new_tokens 2048 \
--vllm_limit_mm_per_prompt '{'image': 3, 'video': 3, 'audio': 3}'
请问vllm版本有什么要求吗?我尝试了0.11和0.9.2都没效果
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported swift infer command with the Qwen3-Omni model, checkpoint, and vLLM backend. Compare the reported vLLM 0.11 and 0.9.2 environments while tracing the LoRA support check that raises the error. Done means identifying a supported configuration or confirming and documenting the limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100