deepseek-ai / deepseek-ai/DeepEP
Maybe use reference stash to replace record stream to reduce mem peak
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
Due to behavior of CudaCacheAllocator, record stream will lead to a late memory free, which has significant on memory peak. (Could refer to [fsdp1's issue due to recordstream](https://dev-discuss.pytorch.org/t/fsdp-cudacachingallocator-an-outsider-newb-perspective/1486)). In Pytorch 2.8 +,[c10d has remove all recordStream in collective communication](https://github.com/pytorch/pytorch/blob/main/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp#L2910). Thus, I wonder if there are plans to remove record stream and use reference stash to handle multistream scenario?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.