mindspore-ai / mindspore-ai/hyper-parallel
门禁 level0 ST 压力过大:NPU 用例全量必跑,需按平台分级整改
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 53
- Forks
- 63
- Avg merge
- 23h 45m
- Merged PRs (30d)
- 63
Description
该问题是怎么引起的?
合入门禁(level0)当前包含 173 个 NPU ST 用例,其中大量为单算子测试、精度/布局/mesh 形态变体等窄面用例,每个 PR 都要全量占用 Ascend 卡资源执行,门禁排队与资源压力过大。而 gloo(cpu_linux)后端用例不占 NPU 卡,适合承担每 PR 的功能看护,但 torch shard core、tensor_parallel、fully_shard、expert_parallel、checkpoint、pipeline_parallel 等模块缺少 gloo 覆盖(测试 impl 硬编码 npu,无法在 CPU 后端运行)。
重现步骤
- 提交任意 PR 触发门禁;
- 观察 level0 ST 阶段:173 个 NPU 用例全量执行,占用整机 Ascend 资源;
- 检查 tests/torch 各模块:除 shard/ops 外基本没有 cpu_linux 平台的 level0 看护用例。
报错信息
无报错,属于工程效率问题。整改预期:
- 每个模块仅保留覆盖面最全的核心 NPU 用例在 level0,其余 NPU 用例降至 level1(每日执行);
- gloo 用例保留在 level0,并为失去 level0 看护的路径补充 gloo 孪生用例(含使能所需的 fully_shard CPU/gloo 支持);
- 门禁 NPU level0 用例数显著下降,CPU/gloo 看护增强,每日 level1 覆盖不变。
schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 203
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/203
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the level0 ST configuration and the tests/torch modules, especially shard/ops and the shard, tensor_parallel, fully_shard, expert_parallel, checkpoint, and pipeline_parallel paths. Identify which of the 173 NPU cases provide the broadest coverage, then compare their gloo CPU support and existing platform-level settings. Done means level0 retains core NPU coverage, missing gloo twins and required fully_shard CPU support are added, and the remaining NPU cases move to level1 without reducing daily coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100