mindspore-ai / mindspore-ai/hyper-parallel
AutoModels routed EP 公共执行逻辑与路由兼容性
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 53
- Forks
- 63
- Avg merge
- 23h 45m
- Merged PRs (30d)
- 63
Description
需求背景
Qwen3-MoE router 存在 logits 和三元组两种返回形式;DeepSeek router 可以直接返回已计算的 indices/weights。EP 适配需要正确处理这些契约,并在 production local region 中保持 local Tensor 输出。
同时,动态图与静态图需要共享路由和专家数学计算,避免分别维护造成语义偏差。
开发设计
- 兼容 Qwen3 的两种 router 返回形式及 norm_topk_prob;直接透传 DeepSeek 原生 top-k indices/weights。
- 提取 compute_swiglu_expert,统一 fused/split 投影及 activation。
- 引入模型局部的 RoutedEPExecutionOp 和 bind_routed_ep_execution,默认继续使用动态 ragged All-to-All;调用方可以绑定其他执行实现。
- production local region 保留 local Tensor,validation 模式保留 DTensor 输出检查。
- 公共层不导入 compile;静态 capture、固定容量通信与混合并行适配由独立 PR !1295 承载。
验收标准
- router 权重语义、生产与验证模式的输出契约有 UT 覆盖。
- fused/split expert 的前向和梯度一致,空 expert 保留零梯度连接。
- 执行绑定支持参数透传、模型隔离、恢复及非法绑定检查。
- 默认动态图执行行为可独立运行。
验证结果
已在独立 editable 安装的公共源码上执行:
HYPER_PARALLEL_PLATFORM=torch PYTHONNOUSERSITE=1 python -m pytest -q tests/ut/dual_mode_dtensor/test_ep_execution.py tests/ut/dual_mode_dtensor/test_ep_moe_local_region.py
结果:16 passed,6 subtests passed,1 条 TypedStorage 弃用警告。
环境:Python 3.10,torch 2.14.0.dev20260719+cpu,torch_npu 2.14.0.dev20260719。
本次 squash 前后代码树一致。公共分支未单独执行多卡 ST 或性能测试;服务端 CI 和 CodeCheck 待 PR 创建后验证。
schema_version: 1
source: gitcode
gitcode_repo: mindspore/hyper-parallel
gitcode_issue: 367
source_url: https://gitcode.com/mindspore/hyper-parallel/issues/367
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/ut/dual_mode_dtensor/test_ep_execution.py and tests/ut/dual_mode_dtensor/test_ep_moe_local_region.py, which cover the stated execution and local-region behavior. Then trace the router contracts, compute_swiglu_expert, RoutedEPExecutionOp, and bind_routed_ep_execution described in the issue. Done means the listed router, expert-gradient, binding, output-contract, and default dynamic-execution criteria are covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100