Duplicated groups for EP and DP_EP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 184
- Avg merge
- 13h 4m
- Merged PRs (30d)
- 1
Description
In MockNcclGroup, this line: https://github.com/aliyun/SimAI/blob/cee5b9f1d3806322e4a9912beb4d7778a93f179c/astra-sim-alibabacloud/astra-sim/system/MockNcclGroup.cc#L109
just repeats the group creation EP times, but keeps creating the same groups (with same ranks), as the j index is not used.
In my understanding, only one iteration should be enough for creating all the needed groups (i.e., _EP_nums).
A similar problem affects the DP_EP groups creation: https://github.com/aliyun/SimAI/blob/cee5b9f1d3806322e4a9912beb4d7778a93f179c/astra-sim-alibabacloud/astra-sim/system/MockNcclGroup.cc#L137
At last, this does not create a problem in the simulation run because there are just redundant groups (and they just keep replacing themselves in the GroupIndex structure, as the key is the same, eventually leaving only the last one there), but it creates a lot of confusion to me.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in astra-sim-alibabacloud/astra-sim/system/MockNcclGroup.cc at the EP group creation near line 109 and the DP_EP creation near line 137. Check how the j index is used and confirm that identical groups are being created repeatedly. Done means redundant group creation is removed while the simulation still creates the required groups and runs as before.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100