Make metrics consume reward_components directly
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 349
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 318
Description
Problem
BaseMultiRewardVerifyResponse defines reward_components as the canonical multi-reward representation, but aggregate metrics and reward profiling only inspect top-level numeric fields.
Verifiers must therefore duplicate every component:
python
{
"reward": 2.0,
"reward_components": {"correctness": 1.0, "format": 1.0},
"correctness": 1.0,
"format": 1.0,
}
This creates two sources of truth.
Proposal
Teach generic metrics consumers to read reward_components directly, including:
Aggregate metrics
Reward profiling and rollout_infos
Pass/majority metrics where applicable
Other code that generically extracts numeric verifier fields
Preserve existing output names such as mean/correctness.
During migration, matching duplicated values should be deduplicated; conflicting values should produce a clear error.
Acceptance criteria
- Per-component metrics work without duplicate top-level fields.
- Scalar-only verifiers remain unchanged.
- Existing unrelated top-level numeric fields still work.
- Missing, inconsistent, and conflicting components have documented behavior.
- Update the multi-reward example, tests, and documentation to remove the duplication workaround.
- NeMo RL already consumes reward_components directly for GDPO, so this aligns evaluation with the training contract.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Search for generic metric consumers, reward profiling and rollout_infos, pass/majority metrics, and numeric verifier-field extraction. Then inspect the multi-reward example, related tests, and documentation. Done means reward_components works without duplicated fields, scalar verifiers and unrelated numeric fields remain compatible, conflicts have clear behavior, and the workaround is removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- analytics, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100