deepseek-ai / deepseek-ai/FlashMLA
Why vcache fix to "None"
- Dominant language
- C++
- Stars
- 12.9k
- Forks
- 1.2k
- Avg merge
- 4h 20m
- Merged PRs (30d)
- 2
Description
https://github.com/deepseek-ai/FlashMLA/blob/480405ada9beff03b3e99a1dc28c9a35deb8a05c/flash_mla/flash_mla_interface.py#L58
Why does _def flash_mla_with_kvcache_ fix vcache to None when calling _flash_mla_cuda.fwd_kvcache_mla_? In the _fwd_kvcache_mla_ C code, when the vcache pointer is null, vcache is equal to kcache.
`at::Tensor vcache = vcache_.has_value() ? vcache_.value() : kcache;`
It seems that only one cache is needed to complete the MLA calculation? However, from the paper and the open source deepseek-v3 library, I did not find that two caches can share a piece of memory. Could you explain this problem? Thanks a lot!!!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.