Issue with metrics of difficulty_mask_ratio and entropy
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 312
- Avg merge
- 1h 2m
- Merged PRs (30d)
- 2
Description
Hi team,
- I've identified an issue where the difficulty_mask_ratio metric gets overwritten during multi-domain training, causing us to lose valuable per-domain information.
In RLVRPipeline, the code calculates the difficulty_mask_ratio for each domain within a loop.
https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/pipeline/rlvr/rlvr_pipeline.py#L419
In this function, this ratio is then logged to the static metric key actor/difficulty_mask_ratio.
https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/utils/functionals.py#L608
Because the same metric key is used for every domain, the value is overwritten in each iteration. As a result, we can only see the difficulty_mask_ratio of the very last domain processed in the logs. I suggest making the metric key domain-specific.
- Also, the entropy is calculated at once https://github.com/alibaba/ROLL/blob/9b85e63ad4c715aa6602ba2a43657e25217c7732/roll/pipeline/rlvr/rlvr_pipeline.py#L401
but it assign to every domain's metrics with the same value, how about calculate every domain's entropy dividely?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in roll/pipeline/rlvr/rlvr_pipeline.py around lines 401 and 419, then inspect roll/utils/functionals.py around line 608. Trace how entropy and difficulty_mask_ratio are assigned inside the per-domain loop. Done means logs retain separate difficulty_mask_ratio values for each domain and entropy is calculated and reported per domain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100