support base model + multi adapter for actor, critic, ref and reward model
Open
feature request
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
support a base model + multi adapter for actor, critic, ref and reward model like [this](https://github.com/lvwerra/trl/pull/373), this will save much memory
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.