aliyun / aliyun/SimAI

测试发现workload存在下面这些模块,ncclFlowModel 输出文件为空。

Open
#248 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
184
Avg merge
13h 4m
Merged PRs (30d)
1

Description

通过测试发现下面这些模块会导致ns3的仿真模拟的输出ncclFlowModel 文件内容为空,请问这个问题如何解决?
attention_column -1 910558 ALLGATHER 335544320 910558 REDUCESCATTER 0 910558 NONE 0 100
attention_row -1 910558 REDUCESCATTER 335544320 910558 ALLGATHER 335544320 910558 NONE 0 100
mlp_column -1 1448385 ALLGATHER 335544320 1448385 REDUCESCATTER 0 1448385 NONE 0 100
mlp_row -1 1448385 REDUCESCATTER 335544320 1448385 ALLGATHER 335544320 1448385 NONE 0 100
final_column -1 910558 ALLGATHER 335544320 910558 REDUCESCATTER 0 910558 NONE 0 100
mlp_moelayer -1 7956367 ALLGATHER 1572864 7956367 ALLGATHER 1572864 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL 25165824 1 ALLTOALL 25165824 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL_EP 100663296 1 ALLTOALL_EP 100663296 1 NONE 0 100
mlp_moelayer -1 1 ALLGATHER 201326592 1 REDUCESCATTER 201326592 1 NONE 0 100
mlp_moelayer -1 1 REDUCESCATTER 201326592 1 ALLGATHER 201326592 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL_EP 100663296 1 ALLTOALL_EP 100663296 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL 25165824 1 ALLTOALL 25165824 1 NONE 0 100

还要一个问题,一个大模型的transformer是叠加的,计算耗时也是一样的。在使用ns3模拟的时候是不是不用完整输入整个workload,只输入其中一个transformer块,计算一个时间,然后在乘以模块数量,这样可以大幅降低模拟时间,这样做是否能保证仿真的正确性?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the empty ncclFlowModel output with the listed workload modules and trace where that output is generated. No source file, test, or entry point is named in the issue, so identify the relevant simulation path before assessing the workload reduction proposal. Done means explaining the empty output and establishing whether simulating one transformer block preserves correct timing and communication behavior.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, machine-learning, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.