alibaba / alibaba/ROLL

Issue with metrics of difficulty_mask_ratio and entropy

Open
#108 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
312
Avg merge
1h 2m
Merged PRs (30d)
2

Description

Hi team,
- I've identified an issue where the difficulty_mask_ratio metric gets overwritten during multi-domain training, causing us to lose valuable per-domain information.
In RLVRPipeline, the code calculates the difficulty_mask_ratio for each domain within a loop.
https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/pipeline/rlvr/rlvr_pipeline.py#L419
In this function, this ratio is then logged to the static metric key actor/difficulty_mask_ratio.
https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/utils/functionals.py#L608
Because the same metric key is used for every domain, the value is overwritten in each iteration. As a result, we can only see the difficulty_mask_ratio of the very last domain processed in the logs. I suggest making the metric key domain-specific.

- Also, the entropy is calculated at once https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/pipeline/rlvr/rlvr_pipeline.py#L401
but it assign to every domain's metrics with the same value, how about calculate every domain's entropy dividely?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in roll/pipeline/rlvr/rlvr_pipeline.py around lines 401 and 419, then inspect roll/utils/functionals.py around line 608. Trace how entropy and difficulty_mask_ratio are assigned inside the per-domain loop. Done means logs retain separate difficulty_mask_ratio values for each domain and entropy is calculated and reported per domain.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.