magi attention support
Open
community-request
enhancement
waiting-on-maintainers
x-mini
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
[MagiAttention](https://github.com/SandAI-org/MagiAttention) has a good support for flexible attention masking designs, and support almost all modern features in long context training. Should we add it as another attention backend?
Contributor guide
Assessment
This issue has not been assessed yet.