brain-score / brain-score/brain-score.web
view scores at specified point in time
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
We're currently always presenting "today's" leaderboard.
But for references, it might often be useful to point at the state of the leaderboard at some point in the past.
For instance, for the 2019 CORnet paper, we might want to point to the model scores at the time. Similarly, a new paper might want to point out its state-of-the-art scores at the time of submission/publication.
Challenges:
* we do not have a timestamp of when we set benchmarks to public, so going back in time might show scores from benchmarks that were private at the time, i.e. did not show up on the leaderboard. If grouping ever changes, then we also do currently not have any way to keep track of that. I wonder if it would be better to just store the entire website each day, rather than doing this via database timestamps
* ideally we'd like to highlight certain points in time, e.g. "2020 Neuron paper", "before 2022 competition" etc. How can we design this?
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by mapping how leaderboard scores, benchmark visibility, grouping, and timestamps are currently represented, then compare the proposed historical views with daily website snapshots. Done means a concrete, agreed design for retrieving and presenting past leaderboard states and highlighting named points in time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- full-stack
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100