modelscope / modelscope/ms-swift
Any way to access entropy metrics in a custom reward function?
Open
Nobody has claimed this yet.
stale
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the feature
I want to add an entropy reward to my reward function to encourage exploration. Since now we can enable the log_entropy option to record entropy-related metrics, is there a way to access these metrics inside the call method of a custom reward function?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing where the log_entropy option is handled and how custom reward functions invoke their call method. Determine whether entropy metrics are available at that point and what interface would expose them. Done means the supported access path, or its limitation, is implemented and covered by appropriate tests or documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100