NVIDIA-NeMo / NVIDIA-NeMo/RL

Custom metrics in reward function

Open
#2,022 5 comments 0 reactions 1 assignee Assigned to @ZhiyuLi-Nvidia View on GitHub
community-request enhancement
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

**Is your feature request related to a problem? Please describe.**
Currently, the reward score is the only metric logged with the logger. The ability to report custom metrics (as well as standard min/max/mean breakdown) would be great.

**Describe the solution you'd like**
The ability for reward functions to return something more than reward, correct. e.g. a dict with `{"reward": ..., "additional_metrics": {...}}` or similar. I believe verl does something similar, expecting `score` (as reward) then additional metrics as all additional kwargs.

**Additional context**
An example here would be if the reward is a composite of multiple metrics, reporting each.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.