deepseek-ai / deepseek-ai/DeepEP
How to reduce the shared memory footprint of the low latency combine kernel?
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
I am encountering a runtime error when running the `internode_ll::combine kernel` with hidden=2048. The error is `cudaFuncSetAttribute` failed, which is caused by the high shared memory requirement of the kernel's send phase approx. (200KB).
```
RuntimeError: Failed: Assertion error DeepEP/csrc/kernels/internode_ll.cu:985 'cudaFuncSetAttribute(combine_func, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_size) == cudaSuccess'
```
I identified three main parameters that control the shared memory size: `hidden`,`kNumStages`,`kNumMaxUnrolls`
To resolve the runtime error, I need to reduce the shared memory allocation. Given that the hidden size is fixed(2048), which of the other two parameters is the intended or safest one to adjust?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.