modelscope / modelscope/mcore-bridge
AttributeError when linear_decoupled_in_proj is enabled
Open
@addsubmuldiv is already working on this.
Since Jun 17, 2026.
bug
- Dominant language
- Python
- Stars
- 101
- Forks
- 43
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 23
Description
Checklist / 检查清单
- I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。
Bug Description / Bug 描述
问题描述
在 linear_decoupled_in_proj 模式下，新创建的线性层（in_proj_qkvz 和 in_proj_ba）是在 super().__init__ 之后才创建的，没有被 DDP 正确处理，导致 main_grad 属性缺失。
当 gradient_accumulation_fusion 启用时，反向传播会访问 weight.main_grad.dtype，但由于 main_grad 是 None，会出现：
AttributeError: 'NoneType' object has no attribute 'dtype'
在 1.4 版本和 Qwen3.5 适配中，gradient_accumulation_fusion 是默认启用的（通过 --no-gradient-accumulation-fusion 可以禁用）。
临时解决通过配置 --no-gradient-accumulation-fusion 来避免这个问题。
How to Reproduce / 如何复现
mindspeed 0.16
megatron-core 0.16
mcore-bridge 1.4
ms-swift 4.2
Ascend 910B
megatron sft qwen3.5 35b-a3b
Additional Information / 补充信息
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.