ByteDance-Seed / ByteDance-Seed/Triton-distributed

Support MLA Decode

Open
#60 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.5k
Forks
172
PR merge metrics
No merged PRs in 30d

Description

Thank you very much for your contributions !!! I noticed that test_sp_decode_attn.py test calls GQA, specifically SpGQAFlashDecodeAttention, is being called. Does it currently support MLA's SP Decode? We hope to integrate MLA's Decode SP into our DeepSeek V3 inference framework. I would be most grateful if the MLA Decode SP function could be supported. @KnowingNothing @houqi

Contributor guide

Open the contributing guide

Research direction

Start with test_sp_decode_attn.py and inspect how SpGQAFlashDecodeAttention is called for GQA. Determine whether the current path supports MLA Decode SP, then compare the required behavior with the DeepSeek V3 inference use case. Done means MLA Decode SP is supported and covered by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.