brain-score / brain-score/brain-score.web
timestamp benchmarkinstance version update
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 7
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
We need to start keeping track of when new benchmarks and new versions are released so that we can provide snapshots of scores and rankings in time.
For instance, if a paper claims SOTA on Brain-Score 2021, we should be able to present the leaderboard with the benchmarks of that year, even after corresponding benchmarks have maybe been changed.
To get started on this, I propose to add a new field `release_timestamp` to [`BenchmarkInstance`](https://github.com/brain-score/brain-score.web/blob/afc5c1a1bf512d8a1f88d4b1c8461001bd13cd42/benchmarks/models.py#L128) that reflects when a particular benchmark (version) was released.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in benchmarks/models.py at BenchmarkInstance and read how its existing fields are defined and persisted. Add the release_timestamp field so each benchmark version can retain its release time; done means the model stores that timestamp for later historical snapshots.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, database
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100