Migrate to `peft` from `opendelta` for parameter efficient tuning methods
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
Let's migrate to [`peft`](https://github.com/huggingface/peft).
##### Tasks
Doing so will require the following updates:
1. Replace the `opendelta` setup in the `AccelerateBaseTrainer` with a `peft` backed setup: https://github.com/CarperAI/trlx/blob/92b68e4d8c5d59e6ba25d12fd9acfe10287be689/trlx/trainer/accelerate_base_trainer.py#L145-L155
2. Handle fine-grained layer capturing to only modify the upper trunk layers of hydra architectures as handled below:
https://github.com/CarperAI/trlx/blob/92b68e4d8c5d59e6ba25d12fd9acfe10287be689/trlx/utils/modeling.py#L414-L428
##### Motivation
Citing @ethankim00's concerns with `opendelta`:
- `opendelta` import fails due to an unnecessary `turtle` package import. Even if pip installed, users may be required to have `sudo` privileges to install the corresponding base graphics package https://github.com/thunlp/OpenDelta/issues/47
- Doesn’t seem to work with DeepSpeed ZeRO 3
- Additional inference overhead from not merging in the LoRA adapters layers
- Incompatibility with int8 training
- Less actively maintained than the `peft` library, which has been growing rapidly
- Sharing adapter weights on the HuggingFace Hub is less convenient with `opendelta`
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.