huggingface / huggingface/alignment-handbook
LoRA + FlashAttention2 speed up？
Open
- Dominant language
- Python
- Stars
- 5.7k
- Forks
- 490
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
When fine-tuning Mistral with LoRA, do you think FlashAttention2 helps in speeding up the process? If yes, how significant is the acceleration? Where is the primary acceleration achieved?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.