mindspore-ai / mindspore-ai/hyper-parallel
【RFC】 分布式算子待规划事项
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 53
- Forks
- 63
- Avg merge
- 23h 45m
- Merged PRs (30d)
- 63
Description
1. 分布式算子流程重构迁移
任务:
- 将存量分布式算子从_with_layout_infer迁移至_dispatch_new新流程,新旧流程通过是否使用preprocess方法区分。
#94
https://gitcode.com/david-he91/docs/blob/main/HyperParallel/op_dispatch_refactor_v2.md
2. 分布式算子开发
任务:
- 当前各网络还缺失部分分布式算子实现未完成,需要继续补齐。
#124
3. 分布式算子UT覆盖补齐
任务:
- 当前分布式算子覆盖率不够,需要补齐覆盖
https://repo.mindspore.cn/mindspore/hyper-parallel/version/202605/20260519/master_20260519103111_51c236bd893d04e7d4325c38d1682b062ab5b78d/coverage/htmlcov
完成情况:
- #159 已合入
4. 分布式算子ST用例整改
任务:
- 对torch平台添加gloo cpu后端用例,补充CI对于基础标准算子的接口功能/精度的基础覆盖
- 探索mindspore平台dryrun跑分布式算子用例,补充CI对于mindspore算子接口功能分布式逻辑的覆盖
- 梳理ST用例编写规则,尽可能少的使用卡,shard_size最大是2等规则重构用例。引入AI对入参覆盖进行分析,增加覆盖,减少重复/无效用例。
完成情况:
5. 分布式算子实现逻辑重构(探索)
任务:
- 参考torch分布式算子的OpSchema,当前分布式算子逻辑抽象的不够,写法较为复杂,性能可能较低。
- Inplace类算子当前的分布式逻辑,注册在ElementWiseDistributedOp上,实际上是冗余且存在潜在风险的。
- ElementWiseDistributedOp分布式逻辑当前实现应用于单元/二元算子,部分涉及广播等场景实现逻辑复杂,可能存在冗余。
- 当前yaml注册中存在无效/未被使用的参数。
6. AI辅助流程完善
任务:
- 分布式算子skill,基于新流程整体重写,当前skill过于厚重,约束太多。
- ascend-test skill,告知如何跑通用例,提供相关约束。后续可以跑CPU的用例之后,可以在plan步骤之后,将人从dev-test-debug的loop中排除。
https://gitcode.com/david-he91/docs/blob/main/HyperParallel/RotaryPositionEmbedding_claude_session.md
schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 156
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/156
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with linked issue #94 and op_dispatch_refactor_v2.md to understand the distributed-operator migration, then review #124, #159, #122, and #158 for related implementation and test work. The issue spans several exploratory workstreams, so completion is not defined here; each selected task needs its own scope and acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ci-cd, devtools, distributed-systems, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100