deepseek-ai / deepseek-ai/DeepEP

[Performance] Question about performance of Deepep v2, the role of the prefer-overlap-with-compute parameter

Open
#669 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

I would like to ask, what are the execution commands for the performance of the two machines shown in the figure below, and do they need to include --prefer-overlap-with-compute 1 ?

Image

I am not aligned with the official performance data in my local environment, and the following are the test results.
[1]python tests/elastic/test_ep.py --num-processes 8 --allow-hybrid-mode 1 --num-sms 12 --reuse-elastic-buffer --num-topk 8 --num-tokens 8192 -**-prefer-overlap-with-compute 1** --num-qps 65

Image

[2]python tests/elastic/test_ep.py --num-processes 8 --allow-hybrid-mode 1 --num-sms 12 --reuse-elastic-buffer --num-topk 8 --num-tokens 8192 -**-prefer-overlap-with-compute 0** --num-qps 65

Image

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.