Lightning-AI / Lightning-AI/litgpt
QLoRA with FSDP Support
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 1.5k
- Avg merge
- 15h 37m
- Merged PRs (30d)
- 1
Description
I see that [PEFT brought in](https://github.com/huggingface/peft/releases/tag/v0.10.0) QLoRA with FSDP support in their latest release.
Any plans to incorporate this into litgpt?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the linked PEFT v0.10.0 release to understand its QLoRA with FSDP support, then compare that scope with litgpt. Done means defining and incorporating the required support so the corresponding QLoRA and FSDP workflow is available in litgpt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100