brain-score / brain-score/brain-score.web
mark models without all scores as incomplete
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
On the current site, e.g. efficientnet-b0 is very low in ranking, but lacks a lot of benchmark scores (e.g. all of Marques2020).
This is confusing and we should clearly mark incomplete models as such.
I propose a yellow warning tag (https://bulma.io/documentation/elements/tag/#colors) under the model name, behind the submitter name, to point out models with incomplete scores. E.g.
> Incomplete: not scored on 27/34 benchmarks
and a tooltip on hover
> Missing benchmark scores are treated as zero when averaging. This might lead to lower average scores and ranking until the model is fully evaluated.
This warning tag should appear on both the leaderboard as well as the model page.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the leaderboard and model page views, then trace how each model's benchmark scores and averages are displayed. Add the incomplete warning with the missing-score count and tooltip in both views, and verify that models lacking scores are clearly marked without changing the existing ranking behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100