deepseek-ai / deepseek-ai/FlashMLA
Missing sparse backward kernel.
Open
- Dominant language
- C++
- Stars
- 12.9k
- Forks
- 1.2k
- Avg merge
- 4h 20m
- Merged PRs (30d)
- 2
Description
Thank you for the excellent work on the highly optimized attention kernels in FlashMLA. The performance of the sparse forward kernel `flash_mla_sparse_fwd` for the prefill stage is particularly impressive.
I've noticed that while the library provides a complete forward and backward pass for dense attention, but the sparse attention implementation appears to be forward-only. **Will the sparse attention backward kernel be released**?
Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.