Confused by config parameters among total_steps, epochs, batch_size and num_rollouts
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 📚 The doc issue
TrainConfig has some general explanation about some of the parameters. But after running the [ppo_hh.py](https://github.com/CarperAI/trlx/blob/main/examples/hh/ppo_hh.py), i got confused.
1. ppo_hh.py set total_steps and epochs while it seems epochs was ignored in the training.
2. How many data will a step consumed? num_rollout or batch_size?
3. In my understanding, every epoch uses num_rollout examples. They are used for generation in a chunk_size batch. Then every step use batch_size to update the policy model. Am i right?
Thank you in advance for answering all the questions.
### Suggest a potential alternative/fix
Maybe there should be some more detailed description about these parameters?
Contributor guide
Assessment
This issue has not been assessed yet.