modelscope / modelscope/ms-swift
可以使用transformer推理lora微调后的模型吗
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the bug
我基于Megatron微调qwen3omni模型，并且用mega_bridge合并了原有的safe_tensor，再使用原有的 transformer就会报 some weights of the model checkpoint at were not initialized from model checkpoint warning，看了下主要是没微调的talker和code2wav部分，推理异常地慢（600s一次推理）
Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
cuda: 12.5
gpu: L20
torch: 2.8.0
Additional context
推理脚本: 同官方的推理例程 https://github.com/QwenLM/Qwen3-Omni
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the official Qwen3-Omni inference example linked in the issue and trace the checkpoint produced by mega_bridge before it is loaded by transformer. Inspect the initialization warnings for the talker and code2wav components and compare the merged model with the original checkpoint. Done means establishing whether this LoRA-merged model is supported and documenting or correcting the loading path if it is not.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100