ByteDance-Seed / ByteDance-Seed/Triton-distributed
Support MLA Decode
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
Thank you very much for your contributions !!! I noticed that test_sp_decode_attn.py test calls GQA, specifically SpGQAFlashDecodeAttention, is being called. Does it currently support MLA's SP Decode? We hope to integrate MLA's Decode SP into our DeepSeek V3 inference framework. I would be most grateful if the MLA Decode SP function could be supported. @KnowingNothing @houqi
Contributor guide
Research direction
Start with test_sp_decode_attn.py and inspect how SpGQAFlashDecodeAttention is called for GQA. Determine whether the current path supports MLA Decode SP, then compare the required behavior with the DeepSeek V3 inference use case. Done means MLA Decode SP is supported and covered by an appropriate test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100