brain-score / brain-score/brain-score.web
exclude incomplete models from comparison plot
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
e.g. comparing IT vs behavior, a chunk of models (efficientnet-b*) were not scored on some of the IT benchmarks, leading to a score of 0 which in turn pushes down the IT average -- I think these models with missing scores should be excluded from the comparison

related to #136
Contributor guide
No contributing guide indexed for this repository
Research direction
No file, test, or entry point is named. Start by locating the comparison plot and the code that turns missing benchmark scores into zeroes; inspect how IT and behavior averages are computed. Done means models missing scores for some benchmarks no longer lower the comparison averages, with the incomplete-model case covered by the relevant check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data-visualization
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100