mindspore-ai / mindspore-ai/hyper-parallel

【RFC】 分布式算子待规划事项

Open
#716 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
53
Forks
63
Avg merge
23h 45m
Merged PRs (30d)
63

Description

1. 分布式算子流程重构迁移
任务:
2. 分布式算子开发
任务:
  • 当前各网络还缺失部分分布式算子实现未完成,需要继续补齐。
    #124
3. 分布式算子UT覆盖补齐
任务:
完成情况:
4. 分布式算子ST用例整改
任务:
  • 对torch平台添加gloo cpu后端用例,补充CI对于基础标准算子的接口功能/精度的基础覆盖
  • 探索mindspore平台dryrun跑分布式算子用例,补充CI对于mindspore算子接口功能分布式逻辑的覆盖
  • 梳理ST用例编写规则,尽可能少的使用卡,shard_size最大是2等规则重构用例。引入AI对入参覆盖进行分析,增加覆盖,减少重复/无效用例。
完成情况:
5. 分布式算子实现逻辑重构(探索)
任务:
  • 参考torch分布式算子的OpSchema,当前分布式算子逻辑抽象的不够,写法较为复杂,性能可能较低。
  • Inplace类算子当前的分布式逻辑,注册在ElementWiseDistributedOp上,实际上是冗余且存在潜在风险的。
  • ElementWiseDistributedOp分布式逻辑当前实现应用于单元/二元算子,部分涉及广播等场景实现逻辑复杂,可能存在冗余。
  • 当前yaml注册中存在无效/未被使用的参数。
6. AI辅助流程完善
任务:

schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 156
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/156

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with linked issue #94 and op_dispatch_refactor_v2.md to understand the distributed-operator migration, then review #124, #159, #122, and #158 for related implementation and test work. The issue spans several exploratory workstreams, so completion is not defined here; each selected task needs its own scope and acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ci-cd, devtools, distributed-systems, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.