deepseek-ai / deepseek-ai/DeepEP
HT Kernels with "--test-ll-compatibility" option
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
The question is in the context of high throughput kernels with the option `--test-ll-compatibility`.
1. What is this compatibility checking? It seems that the compatibility of the HT kernel buffer is checked with LL kernel buffer. Is it in the context of prefill stage followed by a decode stage (in inference)? I understand that prefill will use HT kernels and decode will use decode kernels.
2. With this option, how is the NVSHMEM communicator built? For example, with 2 servers (with 8 GPUs each), do we expect to see 8 communicators built (one per NVLink rank)? We're seeing that [this init method in runtime.cu](https://github.com/deepseek-ai/DeepEP/blob/main/csrc/kernels/runtime.cu#L49) gets called by all 16 GPUs. However, all GPUs on the first server have rank 0 (its' rdma_rank) and all GPUs on the second server have rank 1. This is explained in the figure below. This does not seem correct. Kindly clarify.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.