CarperAI / CarperAI/trlx

reproduce trlx example on wandb

Open
#462 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🐛 Describe the bug

Hello everyone, I am attempting to replicate the trlx example found on Wandb at this link: https://wandb.ai/carperai/summarize_RLHF/reports/Implementing-RLHF-Learning-to-Summarize-with-trlX--VmlldzozMzAwODM2

I have a specific question regarding the evaluation performance of the step 1 sft model. In the provided example, the GPT-J 6B model was used, and the evaluation performance was as follows:

rouge1: 0.33495259362557406
rouge2: 0.12516756897761228
rougeL: 0.2614311397592001
rougeLsum: 0.2613398039055508

However, when I tried using the Pythia-1B model, the evaluation performance was:

rouge1: 0.6174393246459715
rouge2: 0.2168605526286073
rougeL: 0.42690106548725937
rougeLsum: 0.5442461303596511

and for Pythia-70M, the evaluation performance was:

rouge1: 0.5579889287651427
rouge2: 0.16015618115528707
rougeL: 0.3522189070990923
rougeLsum: 0.48252334790510854

Can someone please provide an explanation for why the Rouge scores for the sft-tuned GPT-J 6B model are so low compared to the Pythia models?

### Which trlX version are you using?

followed installation, should be lastest

### Additional system and package information

3.9, P3dn24 ec2 instance

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.