ByteDance-Seed / ByteDance-Seed/FlexPrefill
Qwen2 use GQA, but not use gqa_interleave param in `flex_prefill_attention`
Open
- Dominant language
- Python
- Stars
- 172
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
In `flex_prefill/modules/qwen2/flex_prefill_attention.py`, when calling flex_prefill_attention, you keep `gqa_interleave ` False. Additionally, you don't repeat k/v heads in your qwen2_flex_prefill_attention_forward. So what is the meaning of gqa_interleave param? Should it be used in GQA attention?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.