deepseek-ai / deepseek-ai/DeepEP

Has the currently implemented logfmt quantization been able to optimize the combine operation to reduce time consumption?

Open
#338 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hello, I have implemented a logfmt-8bit kernel based on the logfmt quantization in internode-ll for testing the 8-card environment of intranode H800. The combine bandwidth is 160GB/s, the token*hidden number propagated is 4096 and 7168, and the topk is 8. At this time, the time cost without quantization is 1.9ms. For the quantization operation, if kNumUnrolls is set to 1, the time cost is 0.11ms. So, can we say that a good optimization effect has been achieved?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.