CarperAI / CarperAI/trlx

Generic Flan-T5 RL training

Open
#327 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

Hello,
I tried to use this library to train a FLAN-T5. I have seen that as examples there are sentiment analysis or summarization tasks, but in my case it is a generic generation task. Trying to use my dataset, I tried the config: `ppo_config_cnn_daily.yml`.

The result I got is this, and it seems to be a summarization task:

```
Prompt: "[some text] ... from 1978, excluding finishing years incoming 62"
Output: "years incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming"
```

And I'm using this code with my personal prompts:

```
trlx.train(
reward_fn=reward_fn,
prompts=train_prompts,
eval_prompts=valid_prompts,
config=config
)
```

I have two problems, maybe caused by summarization task:
- First the repetition at the end
- Second, the expected output is very different

Should I change the code or are there generic examples? Thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.