modelscope / modelscope/ms-swift
请问一下，微调Qwen3-Next时，设置--target_modules为"all-linear"能否训练到Qwen3NextGatedDeltaNet部分？
Open
Nobody has claimed this yet.
stale
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
请问一下
微调Qwen3-Next时，设置--target_modules为"all-linear"，能否训练到Qwen3NextGatedDeltaNet部分？
transformers中modeling_qwen3_next.py代码中，Qwen3NextGatedDeltaNet类中存在一个self.conv1d = nn.Conv1d部分。
这个self.conv1d中的参数能否在lora训练模式下，通过设置--target_modules为"all-linear"而被训练？
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with transformers' modeling_qwen3_next.py and the Qwen3NextGatedDeltaNet class. Trace how --target_modules="all-linear" is handled in LoRA training and whether self.conv1d is selected. Done means documenting whether those parameters are trained and identifying any required support or configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100