deepseek-ai / deepseek-ai/DeepEP

Maybe use reference stash to replace record stream to reduce mem peak

Open
#455 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Due to behavior of CudaCacheAllocator, record stream will lead to a late memory free, which has significant on memory peak. (Could refer to [fsdp1's issue due to recordstream](https://dev-discuss.pytorch.org/t/fsdp-cudacachingallocator-an-outsider-newb-perspective/1486)). In Pytorch 2.8 +,[c10d has remove all recordStream in collective communication](https://github.com/pytorch/pytorch/blob/main/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp#L2910). Thus, I wonder if there are plans to remove record stream and use reference stash to handle multistream scenario?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.