linkedin / linkedin/Liger-Kernel
Grouped Latent Attention
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 603
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 47
Description
🚀 The feature, motivation and pitch
New work from Prof. Dao's lab that improves on Deepseek's original Multihead Latent Attention.
Relevant Paper: https://arxiv.org/pdf/2505.21487
Alternatives
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry point; begin by reading the linked paper and locating existing attention-kernel implementations in the repository. Before coding, define the integration point and validation needed for grouped latent attention, since the issue does not specify what done looks like.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100