AnswerDotAI / AnswerDotAI/fsdp_qlora
BOFT support?
Open
- Dominant language
- Jupyter Notebook
- Stars
- 1.6k
- Forks
- 201
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Considering the PEFT library has support for the OFT/BOFT adapter, can this be supported in fsdp_qlora too? Would be an useful adapter to have due to its resistance against catastrophic forgetting.
Thanks
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the PEFT library's OFT/BOFT support and comparing it with how fsdp_qlora currently handles adapters. The issue names no files, tests, or entry points, so first identify the adapter integration path. Done would mean confirmed OFT/BOFT support in fsdp_qlora, with appropriate validation.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100