brain-score / brain-score/brain-score.web

better explain how scores are computed / which columns can be compared

Open
#139 0 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
JavaScript
Stars
3
Forks
7
Avg merge
7m
Merged PRs (30d)
1

Description

The scores of different benchmarks are computed in different ways (e.g. distributional match for Marques2020, r^2 for neural predictivity, r for Rajalingham2018-i2n). This is not immediately clear from the website, which can make users compare scores across columns without being aware of the differences.

(from @AlinaPeter, @kohitij-kar, @ratanmurty)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the website areas that display benchmark scores and identify where score computation and comparability are currently described. Document the stated methods, including distributional match, r^2, and r, and clarify which columns should not be compared. Done means users can distinguish the metrics before comparing scores.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.