Generic Flan-T5 RL training
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I tried to use this library to train a FLAN-T5. I have seen that as examples there are sentiment analysis or summarization tasks, but in my case it is a generic generation task. Trying to use my dataset, I tried the config: `ppo_config_cnn_daily.yml`.
The result I got is this, and it seems to be a summarization task:
```
Prompt: "[some text] ... from 1978, excluding finishing years incoming 62"
Output: "years incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming"
```
And I'm using this code with my personal prompts:
```
trlx.train(
reward_fn=reward_fn,
prompts=train_prompts,
eval_prompts=valid_prompts,
config=config
)
```
I have two problems, maybe caused by summarization task:
- First the repetition at the end
- Second, the expected output is very different
Should I change the code or are there generic examples? Thanks
Contributor guide
Assessment
This issue has not been assessed yet.