RL Bug/Issue Tracker
Open
type:bug
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 345
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 240
Description
This issue is a running tracker of all RL bugs/issues we identify:
- [ ] Logging for metrics returned by loss functions via `aux` is at a `self._iter_steps` level, whereas other metrics are logged at a `self.global_steps` level.
- [ ] The progress bar for GRPO/PPO is at a `self._iter_steps` level. We should possibly unify this with how we log metrics. It's a bit confusing for the user to pass `max_steps` at a `self._iter_steps`, and not at `self.global_steps` level.
Contributor guide
Assessment
This issue has not been assessed yet.