modelscope / modelscope/ms-swift
支持RLPR算法（一种RLVR Free的方式）的Reward Func
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
RLPR的论文：https://arxiv.org/abs/2506.18254
核心点：需要在奖励函数里面调用当前的策略模型进行推理，计算question+thinking+label的logps，也需要计算question+label的logps。貌似咱们目前的自定义奖励函数里面无法使用当前策略模型进行计算，希望能考虑支持一下，谢谢。
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked RLPR paper and reviewing how custom reward functions are currently handled. Trace whether the current policy model is available there and how log probabilities are computed. Done would mean custom rewards can calculate both question+thinking+label and question+label logps using the current policy model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100