deepseek-ai / deepseek-ai/FlashMLA

Question of O_accum and LSE_accum shape

Open
#70 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
12.9k
Forks
1.2k
Avg merge
4h 20m
Merged PRs (30d)
2

Description

https://github.com/deepseek-ai/FlashMLA/blob/b31bfe72a83ea205467b3271a5845440a03ed7cb/csrc/flash_api.cpp#L184

Hi all,

Just wondering why the shape of O_accum and LSE_accum change from [numsplit, batch, head, seqlen_q, (dim)] to [**batch + sm_part**, ... ] I can get the idea that sm_parts should be the first dimension of output cz it replaced the batchsize, and I also get that sm_parts is not the exact number of loops cz some batch are merge into one sm_parts, but why is the addition work here?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.