[Tracking Issue] Need support for GQA Attention in Relax
Open
needs-triage
type:rfc-tracking
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 112
Description
This feature is critical for modern LLM compilation workflows and is currently not available in Relax. Adding native support for GQA in Relax will enable better performance and compatibility with transformer-based models exported from PyTorch, HuggingFace, or ONNX formats.
https://arxiv.org/abs/2305.13245
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked GQA paper and the Relax entry point, then trace how transformer models from PyTorch, HuggingFace, or ONNX are represented. Done means Relax has native GQA Attention support for those compilation workflows, with the intended behavior and performance compatibility established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python, pytorch
- Domain
- compilers, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100