AI-Hypercomputer / AI-Hypercomputer/maxtext
Support LoRA training
Open
feature request
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 607
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 158
Description
Is there a plan to support PEFT methods like LoRA training in maxtext to support larger model fine-tuning / continue pretraining so that bigger models like LLaMA-3-70B can be trainined even with small amount of TPU/GPUs?
Contributor guide
Assessment
This issue has not been assessed yet.