huggingface / huggingface/trl

RLOOTrainer & PPOv2Trainer - Modify Name for W&B Logged Table

Open
#2,045 6 comments 0 reactions 0 assignees View on GitHub
✨ enhancement 🏋 PPO 🏋 RLOO 🙋 help from community wanted
Dominant language
Python
Stars
19.3k
Forks
3k
Avg merge
1d 20h
Merged PRs (30d)
194

Description

### Feature request

[RLOOTrainer](https://github.com/huggingface/trl/blob/main/trl/trainer/rloo_trainer.py#L511-L512) and [PPOv2Trainer](https://github.com/huggingface/trl/blob/main/trl/trainer/ppov2_trainer.py#L613-L614) both log a table to W&B titled `"completions"`. I would like the name to be changed to `f"completions_step={self.state.global_step}"` or something like this.

### Motivation

Due to how W&B displays tables in the WebUI, logging the table under the same name `"completions"` means that one can only see the most recent table of completions, rather than tables of completions at different points during learning.

I would like to be able to manually inspect how completions are changing during learning.

I can probably pull this information out of the W&B API, but this is too much effort. I want to be able to manually inspect easily.

### Your contribution

I would be happy to submit a PR for this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.