reward_fn in accelerate_ppo_trainer.py
Open
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
Hi, may I know why these two reward_fn function differently while they seem to be the same one passed to the PPO trainer as input? In my understanding of PPO, the reward function should output rewards for each sample instead of a sequence of (sequence_length, ) rewards.
https://github.com/CarperAI/trlx/blob/3340c2f3a56d1d14fdd5f13ad575121fa26b6d92/trlx/trainer/accelerate_ppo_trainer.py#L309-L310
https://github.com/CarperAI/trlx/blob/3340c2f3a56d1d14fdd5f13ad575121fa26b6d92/trlx/trlx.py#L38-L40
Contributor guide
Assessment
This issue has not been assessed yet.