CarperAI / CarperAI/trlx

T5 FLAN 11B Config for a consumer GPU

Open
#173 3 comments 7 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🚀 The feature, motivation, and pitch

Now that @jon-tow @ethankim00 have merged LoRA and 8bit adam, we can create an example where one RLHF tunes T5 FLAN 11B on a consumer GPU with minimal CPU offloading.

If we can get sentiment ppo working in a 24GB of VRAM constrained environment (like a 3090), I think that would be a great demo to show people who want to run trlX at home.

@reciprocated said that he has gotten CPU offloading working before.

### Alternatives

We could also do 6B with no offloading on a 3090 now as well probably.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.