modelscope / modelscope/ms-swift
DPO和GRPO中的beta作用
Open
Nobody has claimed this yet.
stale
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
操作文档中描述：beta: 控制与参考模型偏差程度的参数。beta值越高，表示与参考模型的偏差越小。
但是实际情况，在dpo和grpo中，从某种程度来看，beta的作用不是相反嘛，dpo中beta越大，越剧烈；而在grpo中beta越大，约束越大，想了解一下是在代码中优化了吗？
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named in the issue. Start by locating the operation documentation and the DPO and GRPO beta handling, then compare the documented meaning with the implementation. Done means resolving whether the descriptions are consistent and updating the documentation or reporting the discrepancy clearly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100