mindspore-ai / mindspore-ai/hyper-parallel
[Feature]: hyper-parallel 注册 npu_mhc_pre_cmhc 融合算子并接入框架
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 53
- Forks
- 63
- Avg merge
- 23h 45m
- Merged PRs (30d)
- 63
Description
🚀 功能描述
需求背景
非融合 HyperConnectionModule 已支持 CMHC 路径(enable_cmhc=True,置换-softmax 混合投影),但融合路径 FusedHyperConnectionModule 只实现了 Sinkhorn,不支持 CMHC。当 use_fused_mhc=True + enable_cmhc=True 时会回退到非融合路径,无法享受融合算子的性能收益。
需求目标
新增 CMHC 前处理融合算子 npu_mhc_pre_cmhc 并接入 hyper-parallel,使 use_fused_mhc=True + enable_cmhc=True 组合能走融合 CMHC 路径,打通 ops-transformer 算子层 → hyper-parallel DFunction → mindformers 调用链。
改动范围
- 新增 AscendC kernel
mhc_pre_cmhc.cc(前向)+mhc_pre_cmhc_backward.cc(反向),通过AclnnOpRunner启动 aclnn - 注册
NpuMhcPreCmhcDFunction(custom_op_impl.py),实现 forward/backward,算子自带 bprop - 导出公开 API
npu_mhc_pre_cmhc(experimental_ops.py+experimental/__init__.py) - 新增分布式算子
parallel_mhc_pre_cmhc(core/shard/ops/)+ yaml 注册 - torch 侧
__init__.py同步导出
关键约束
n(残差流数量)当前仅支持 4gamma保持为 required tensor(nullptr会让 aclnn 运行时崩溃)- ACLNN ABI 详见
ops-transformer/mhc/mhc_pre_cmhc/
现有替代方案
Thanks for contributing 🎉!
与DFX相关性DF
Thanks for contributing 🎉!
提议的新API
Thanks for contributing 🎉!
是否影响现有API
Thanks for contributing 🎉!
补充信息
Thanks for contributing 🎉!
Thanks for contributing 🎉!
schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 317
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/317
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ACLNN ABI in ops-transformer/mhc/mhc_pre_cmhc/, then trace the integration points in custom_op_impl.py, experimental_ops.py, experimental/init.py, core/shard/ops/, the YAML registration, and torch init.py. Done means forward and backward CMHC support is registered across this chain, with n=4 and required gamma handling preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python, pytorch
- Domain
- backend-api-design, distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100