mindspore-ai / mindspore-ai/hyper-parallel

[Feature]: 重构 Trainer Profiling 配置与采集流程

Open
#794 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
53
Forks
63
Avg merge
23h 45m
Merged PRs (30d)
63

Description

🚀 功能描述

当前 Trainer profiling 实现配置能力有限,仅支持单 rank 和基础 PyTorch
Profiler 参数,与MindFormers等生产级训练框架的profiler 生命周期、输出结构等使用方式
存在较大差异。

希望重构 Trainer profiling,提供统一的配置入口,支持按训练步、rank、PP stage、
采集等级、memory、stack 和 MSTX 等维度控制性能采集,并确保 profiler 在采集窗口
结束后及时停止和释放。

现有替代方案

现有 ProfilingCallbacktrainer.runtime.profiling 可以完成基础采集,但扩展能力
有限,继续在旧实现上叠加功能会保留两套配置语义并增加维护成本,因此计划直接替换
旧实现。

与DFX相关性DF
提议的新API

将配置入口统一为 TrainerConfig.profiler,主要支持:

  • enabled
  • start_step / stop_step
  • start_on_init
  • memory
  • rank_ids
  • pipeline_stage_leaders
  • output_path
  • level
  • with_stack
  • data_simplification
  • mstx
是否影响现有API

有影响。本次改动将删除旧 profiling 配置、ProfilingCallback
trainer.runtime.profiling,不提供旧接口兼容层。

示例 YAML、CLI dotted override 和 Trainer callback 初始化逻辑需要同步更新。

补充信息

Thanks for contributing 🎉!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing TrainerConfig.profiler, the existing ProfilingCallback, and trainer.runtime.profiling. Review how YAML examples, CLI dotted overrides, and Trainer callback initialization currently configure profiling. Done means the old configuration and interfaces are removed, the unified options cover the listed collection controls, and profiling stops and releases resources after its window.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.