deepseek-ai / deepseek-ai/FlashMLA

[Question] What is MODEL1?

Open
#155 7 comments 27 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
12.9k
Forks
1.2k
Avg merge
4h 20m
Merged PRs (30d)
2

Description

I found the term "MODEL1" is mentioned several times in this repository:
https://github.com/deepseek-ai/FlashMLA/blob/48c6dc426f045cb7743b18f5c7329f35f1b7ed79/csrc/sm100/decode/head64/kernel.cuh#L554
https://github.com/deepseek-ai/FlashMLA/blob/48c6dc426f045cb7743b18f5c7329f35f1b7ed79/csrc/sm90/decode/sparse_fp8/splitkv_mla.cuh#L161

I have some questions about this novel term. @interestingLSY Could you please take a look?

1. Are there some publicly available models belonging to the category of "MODEL1"?

2. What are the major differences of attention mechanisms between "MODEL1" and V3.2 ?

Update: My preliminary guess is the following for the 2nd question:

1. The topk length becomes dynamic in MODEL1 while constant(2048) in V3.2
2. MODEL1 introduces additional KV states which are not generated from history tokens in the current sequence
3. The number of features in KV cache without position encoding for MODEL1 is 448, which is reduced from 512 as in V3.2
4. Position encoding is also included when recovering V cache

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.