open-compass / open-compass/opencompass
[Feature] Can we find the config version that is used to produce the score displayed on leaderboard?
Open
@tonysy is already working on this.
Since Jan 19, 2024.
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 869
- Avg merge
- 17h 52m
- Merged PRs (30d)
- 13
Description
Describe the feature
Hi, thanks for sharing this wonderful project!
I am wondering is there somewhere we can find the config version that is used to produce the score displayed on leaderboard?
For example, there are config files from different versions under configs/datasets/ARC_c. How should I choose the right version to reproduce the displayed results on leaderboard?
Any method to obtain the right version information would be appreciated!
Will you implement it?
- I would like to implement this feature and create a PR!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.