[QUESTION] why batch_p2p_sync is still forced?
Open
question
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
Is there still a bug in pytorch related to batch_p2p_sync? (Comment says it is a bug in older version of pytorch.)
batch_p2p_sync=True is forced now as there is no argument to set batch_p2p_sync and the default of it is "True" in the following code (https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/model_parallel_config.py#L270)
Thanks
Contributor guide
Assessment
This issue has not been assessed yet.