mindspore-ai / mindspore-ai/hyper-parallel

[Feature]: hyper-parallel 注册 npu_mhc_pre_cmhc 融合算子并接入框架

Open
#651 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
53
Forks
63
Avg merge
23h 45m
Merged PRs (30d)
63

Description

🚀 功能描述

需求背景

非融合 HyperConnectionModule 已支持 CMHC 路径(enable_cmhc=True,置换-softmax 混合投影),但融合路径 FusedHyperConnectionModule 只实现了 Sinkhorn,不支持 CMHC。当 use_fused_mhc=True + enable_cmhc=True 时会回退到非融合路径,无法享受融合算子的性能收益。

需求目标

新增 CMHC 前处理融合算子 npu_mhc_pre_cmhc 并接入 hyper-parallel,使 use_fused_mhc=True + enable_cmhc=True 组合能走融合 CMHC 路径,打通 ops-transformer 算子层 → hyper-parallel DFunction → mindformers 调用链。

改动范围

  • 新增 AscendC kernel mhc_pre_cmhc.cc(前向)+ mhc_pre_cmhc_backward.cc(反向),通过 AclnnOpRunner 启动 aclnn
  • 注册 NpuMhcPreCmhcDFunctioncustom_op_impl.py),实现 forward/backward,算子自带 bprop
  • 导出公开 API npu_mhc_pre_cmhcexperimental_ops.py + experimental/__init__.py
  • 新增分布式算子 parallel_mhc_pre_cmhccore/shard/ops/)+ yaml 注册
  • torch 侧 __init__.py 同步导出

关键约束

  • n(残差流数量)当前仅支持 4
  • gamma 保持为 required tensor(nullptr 会让 aclnn 运行时崩溃)
  • ACLNN ABI 详见 ops-transformer/mhc/mhc_pre_cmhc/
现有替代方案

Thanks for contributing 🎉!

与DFX相关性DF

Thanks for contributing 🎉!

提议的新API

Thanks for contributing 🎉!

是否影响现有API

Thanks for contributing 🎉!

补充信息

Thanks for contributing 🎉!

Thanks for contributing 🎉!

schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 317
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/317

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ACLNN ABI under ops-transformer/mhc/mhc_pre_cmhc/, then trace the related entries in mhc_pre_cmhc.cc, mhc_pre_cmhc_backward.cc, and custom_op_impl.py. Check experimental_ops.py, experimental/init.py, core/shard/ops/, the YAML registration, and the torch-side init.py. Done means the npu_mhc_pre_cmhc API and distributed path are registered across the stated call chain, with n=4 and required gamma preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.