deepseek-ai / deepseek-ai/DeepEP

Unexpected Performance Drop Caused by Explicit deep_ep.Buffer Memory Release

Open
#571 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

## Problem description

Currently, we explicitly release `deep_ep.Buffer` by calling `del buffer` and setting `buffer = None` in order to free device memory:

```python
buffer = deep_ep.Buffer(
group,
int(2e9),
int(1e9),
low_latency_mode=args.test_ll_compatibility,
num_qps_per_rank=num_qps_per_rank,
explicitly_destroy=False,
)
assert num_local_ranks == 8 and num_ranks > 8

del buffer
buffer = None
```
However, this approach forces the buffer to be fully re-created every time it is used again. After upgrading to a newer NVSHMEM version, this repeated creation and destruction introduces a noticeable performance regression, likely due to the increased overhead in buffer initialization and NVSHMEM internal setup.

We would like to know:

- Is there a recommended way to explicitly release or recycle deep_ep.Buffer memory without destroying the entire buffer object?
- Are there any best practices when using deep_ep.Buffer with newer NVSHMEM versions to mitigate this performance drop?

Any guidance or suggestions would be greatly appreciated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.