T5 FLAN 11B Config for a consumer GPU
Open
feature request
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
Now that @jon-tow @ethankim00 have merged LoRA and 8bit adam, we can create an example where one RLHF tunes T5 FLAN 11B on a consumer GPU with minimal CPU offloading.
If we can get sentiment ppo working in a 24GB of VRAM constrained environment (like a 3090), I think that would be a great demo to show people who want to run trlX at home.
@reciprocated said that he has gotten CPU offloading working before.
### Alternatives
We could also do 6B with no offloading on a 3090 now as well probably.
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.