CarperAI / CarperAI/trlx

RLHF text summarization diverges

Open
#589 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🐛 Describe the bug

I am running the experiment of `trlx_gptj_text_summarization.py`, I have not modified the code but the experiment has not converged when more than 3500 steps, and the document said it was meant to converge. I realized the sample project was running the file `trlx_gptneo_text_summarization.py`, but I cannot find the file anywhere.
image

### Which trlX version are you using?

download with source code on 2024/01/13

### Additional system and package information

linux jammy, torch==2.0.0+cu118

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.