deepseek-ai / deepseek-ai/FlashMLA

Missing sparse backward kernel.

Open
#107 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
12.9k
Forks
1.2k
Avg merge
4h 20m
Merged PRs (30d)
2

Description

Thank you for the excellent work on the highly optimized attention kernels in FlashMLA. The performance of the sparse forward kernel `flash_mla_sparse_fwd` for the prefill stage is particularly impressive.

I've noticed that while the library provides a complete forward and backward pass for dense attention, but the sparse attention implementation appears to be forward-only. **Will the sparse attention backward kernel be released**?

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.