请问有支持PoSE的打算吗？
Open
feature request
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
PoSE：https://arxiv.org/abs/2309.10400
https://github.com/OpenAccess-AI-Collective/axolotl/pull/1567
不需要全长度微调，无需修改 attention
好像可以用短上下文训练长上下文
Contributor guide
Assessment
This issue has not been assessed yet.