deepseek-ai / deepseek-ai/DeepEP

Could normal kernel and ll kernel be executed in same process?

Open
#76 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hi, we just tried to integrate DeepEp in our engine. As we supposed, without PD disaggregated mode, we do prefilling with normal mode kernels while do decoding with ll kernels.
But we got crash when we try to initiate two Buffers with different modes, like this:

` self._buffer = get_buffer_normal(group, hidden_size * WEIGHT_DTYPE_SIZE)`
` self._low_latency_buffer = get_buffer_low_latency(group, MAX_DISPATCH_TOKENS_PER_RANK, hidden_size, num_local_experts)`

Some error messages would be given out like below:

![Image](https://github.com/user-attachments/assets/9fcbdc70-2585-4484-a72c-548cba0b5789)

or if we change the statement order, we'll get assertion on this line:

![Image](https://github.com/user-attachments/assets/4dee66b2-e6c5-4f85-9482-c9ab8645bfaf)

So is it by-design that the two mode could not work together in same process? Or is there any advise for our use case? Thx!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.