mindspore-ai / mindspore-ai/hyper-parallel

门禁 level0 ST 压力过大:NPU 用例全量必跑,需按平台分级整改

Open
#263 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
53
Forks
63
Avg merge
23h 45m
Merged PRs (30d)
63

Description

该问题是怎么引起的?

合入门禁(level0)当前包含 173 个 NPU ST 用例,其中大量为单算子测试、精度/布局/mesh 形态变体等窄面用例,每个 PR 都要全量占用 Ascend 卡资源执行,门禁排队与资源压力过大。而 gloo(cpu_linux)后端用例不占 NPU 卡,适合承担每 PR 的功能看护,但 torch shard core、tensor_parallel、fully_shard、expert_parallel、checkpoint、pipeline_parallel 等模块缺少 gloo 覆盖(测试 impl 硬编码 npu,无法在 CPU 后端运行)。

重现步骤
  1. 提交任意 PR 触发门禁;
  2. 观察 level0 ST 阶段:173 个 NPU 用例全量执行,占用整机 Ascend 资源;
  3. 检查 tests/torch 各模块:除 shard/ops 外基本没有 cpu_linux 平台的 level0 看护用例。
报错信息

无报错,属于工程效率问题。整改预期:

  • 每个模块仅保留覆盖面最全的核心 NPU 用例在 level0,其余 NPU 用例降至 level1(每日执行);
  • gloo 用例保留在 level0,并为失去 level0 看护的路径补充 gloo 孪生用例(含使能所需的 fully_shard CPU/gloo 支持);
  • 门禁 NPU level0 用例数显著下降,CPU/gloo 看护增强,每日 level1 覆盖不变。

schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 203
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/203

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the level0 ST configuration and the tests/torch modules, especially shard/ops and the shard, tensor_parallel, fully_shard, expert_parallel, checkpoint, and pipeline_parallel paths. Identify which of the 173 NPU cases provide the broadest coverage, then compare their gloo CPU support and existing platform-level settings. Done means level0 retains core NPU coverage, missing gloo twins and required fully_shard CPU support are added, and the remaining NPU cases move to level1 without reducing daily coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.