deepseek-ai / deepseek-ai/FlashMLA

Why vcache fix to "None"

Open
#46 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
12.9k
Forks
1.2k
Avg merge
4h 20m
Merged PRs (30d)
2

Description

https://github.com/deepseek-ai/FlashMLA/blob/480405ada9beff03b3e99a1dc28c9a35deb8a05c/flash_mla/flash_mla_interface.py#L58

Why does _def flash_mla_with_kvcache_ fix vcache to None when calling _flash_mla_cuda.fwd_kvcache_mla_? In the _fwd_kvcache_mla_ C code, when the vcache pointer is null, vcache is equal to kcache.
`at::Tensor vcache = vcache_.has_value() ? vcache_.value() : kcache;`
It seems that only one cache is needed to complete the MLA calculation? However, from the paper and the open source deepseek-v3 library, I did not find that two caches can share a piece of memory. Could you explain this problem? Thanks a lot!!!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.