brain-score / brain-score/brain-score.web

mark models without all scores as incomplete

Open
#136 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
3
Forks
7
Avg merge
7m
Merged PRs (30d)
1

Description

On the current site, e.g. efficientnet-b0 is very low in ranking, but lacks a lot of benchmark scores (e.g. all of Marques2020).
This is confusing and we should clearly mark incomplete models as such.

I propose a yellow warning tag (https://bulma.io/documentation/elements/tag/#colors) under the model name, behind the submitter name, to point out models with incomplete scores. E.g.
> Incomplete: not scored on 27/34 benchmarks

and a tooltip on hover
> Missing benchmark scores are treated as zero when averaging. This might lead to lower average scores and ranking until the model is fully evaluated.

This warning tag should appear on both the leaderboard as well as the model page.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the leaderboard and model page views, then trace how each model's benchmark scores and averages are displayed. Add the incomplete warning with the missing-score count and tooltip in both views, and verify that models lacking scores are clearly marked without changing the existing ranking behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
frontend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.