deepseek-ai / deepseek-ai/DeepEP
Could normal kernel and ll kernel be executed in same process?
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
Hi, we just tried to integrate DeepEp in our engine. As we supposed, without PD disaggregated mode, we do prefilling with normal mode kernels while do decoding with ll kernels.
But we got crash when we try to initiate two Buffers with different modes, like this:
` self._buffer = get_buffer_normal(group, hidden_size * WEIGHT_DTYPE_SIZE)`
` self._low_latency_buffer = get_buffer_low_latency(group, MAX_DISPATCH_TOKENS_PER_RANK, hidden_size, num_local_experts)`
Some error messages would be given out like below:

or if we change the statement order, we'll get assertion on this line:

So is it by-design that the two mode could not work together in same process? Or is there any advise for our use case? Thx!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.