brain-score / brain-score/brain-score.web
better explain how scores are computed / which columns can be compared
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
The scores of different benchmarks are computed in different ways (e.g. distributional match for Marques2020, r^2 for neural predictivity, r for Rajalingham2018-i2n). This is not immediately clear from the website, which can make users compare scores across columns without being aware of the differences.
(from @AlinaPeter, @kohitij-kar, @ratanmurty)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the website areas that display benchmark scores and identify where score computation and comparability are currently described. Document the stated methods, including distributional match, r^2, and r, and clarify which columns should not be compared. Done means users can distinguish the metrics before comparing scores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100