测试发现workload存在下面这些模块,ncclFlowModel 输出文件为空。
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 184
- Avg merge
- 13h 4m
- Merged PRs (30d)
- 1
Description
通过测试发现下面这些模块会导致ns3的仿真模拟的输出ncclFlowModel 文件内容为空,请问这个问题如何解决?
attention_column -1 910558 ALLGATHER 335544320 910558 REDUCESCATTER 0 910558 NONE 0 100
attention_row -1 910558 REDUCESCATTER 335544320 910558 ALLGATHER 335544320 910558 NONE 0 100
mlp_column -1 1448385 ALLGATHER 335544320 1448385 REDUCESCATTER 0 1448385 NONE 0 100
mlp_row -1 1448385 REDUCESCATTER 335544320 1448385 ALLGATHER 335544320 1448385 NONE 0 100
final_column -1 910558 ALLGATHER 335544320 910558 REDUCESCATTER 0 910558 NONE 0 100
mlp_moelayer -1 7956367 ALLGATHER 1572864 7956367 ALLGATHER 1572864 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL 25165824 1 ALLTOALL 25165824 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL_EP 100663296 1 ALLTOALL_EP 100663296 1 NONE 0 100
mlp_moelayer -1 1 ALLGATHER 201326592 1 REDUCESCATTER 201326592 1 NONE 0 100
mlp_moelayer -1 1 REDUCESCATTER 201326592 1 ALLGATHER 201326592 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL_EP 100663296 1 ALLTOALL_EP 100663296 1 NONE 0 100
mlp_moelayer -1 1 ALLTOALL 25165824 1 ALLTOALL 25165824 1 NONE 0 100
还要一个问题,一个大模型的transformer是叠加的,计算耗时也是一样的。在使用ns3模拟的时候是不是不用完整输入整个workload,只输入其中一个transformer块,计算一个时间,然后在乘以模块数量,这样可以大幅降低模拟时间,这样做是否能保证仿真的正确性?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the empty ncclFlowModel output with the listed workload modules and trace where that output is generated. No source file, test, or entry point is named in the issue, so identify the relevant simulation path before assessing the workload reduction proposal. Done means explaining the empty output and establishing whether simulating one transformer block preserves correct timing and communication behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- distributed-systems, machine-learning, networking
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100