deepspeedai / deepspeedai/DeepSpeed
[BUG]DeepSpeed Comm. Backend not compatible with outside torch.distributed module
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Describe the bug
Recently I have been integrating DeepSpeed into Megatron v3.
However I find that the new DeepSpeed Comm. Backend introduced in the next pr seems not compatible with the outside torch.distributed module. Is there any plan for the compatibility for that?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing DeepSpeed Comm. Backend v1 PR #1985 and the reported integration with Megatron v3. Compare the backend's behavior with the outside torch.distributed module. Done means the compatibility gap is reproduced, its expected behavior is agreed, and a regression check demonstrates compatible communication.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100