[Feature] rotary代码规范化
Open
enhancement
- Dominant language
- Python
- Stars
- 421
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
### Describe the feature
目前 rotary_embedding类的实现有大量历史遗留的代码，建议和https://github.com/Dao-AILab/flash-attention/blob/v2.2.1/flash_attn/layers/rotary.py 对齐，并且支持triton算子
### Will you implement it?
- [ ] I would like to implement this feature and create a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.