modelscope / modelscope/mcore-bridge
训练qwen3.5moe系列时 swift的 moe_grouped_gemm需要为true
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 101
- Forks
- 43
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 23
Description
Checklist / 检查清单
- I have searched existing issues, and this is a new feature request. / 我已经搜索过现有的 issues，确认这是一个新的 Feature Request。
Feature Request Description / Feature Request 描述
训练qwen3.5moe系列时
如果用了mcore-bridge
swift的 moe_grouped_gemm需要为true
虽然这个参数默认就是true，不过置成flase的话 megatron会走SequentialMLP的路子，就会出现和mcore-bridge不兼容～
可以在readme/swift的文档里mark一下
Pull Request / Pull Request 信息
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the README or Swift-related documentation mentioned by the issue. Add a note that Qwen3.5 MoE training with mcore-bridge requires Swift's moe_grouped_gemm to remain true, and explain that setting it false selects an incompatible Megatron path. Done means the requirement and incompatibility are clearly documented.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100