AllGather Message Size vs Workload TXT File Msg Size
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 184
- Avg merge
- 13h 4m
- Merged PRs (30d)
- 1
Description
Hi,
The workload file specifies an AllGather communication with message size computed based on the total params of the model. I am assuming this is the size of the data received at each node after gathering?
The simulator reads this message size generates AllGather using it as initial data size. The final data received at each node becomes number of ranks multipled by the initial message size specified in the TXT workload.
Is this intended? I am wondering whether the AllGather specified in the TXT file needs a division by the number of ranks (world_size).
Please let me know.
--
Vamsi
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the AllGather message size is read from the workload TXT file and how the simulator generates the communication across ranks. Compare the configured initial size with the final per-node received size, then confirm which interpretation matches the intended workload semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100