brain-score / brain-score/brain-score.web

timestamp benchmarkinstance version update

Open
#97 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
3
Forks
7
Avg merge
7m
Merged PRs (30d)
1

Description

We need to start keeping track of when new benchmarks and new versions are released so that we can provide snapshots of scores and rankings in time.
For instance, if a paper claims SOTA on Brain-Score 2021, we should be able to present the leaderboard with the benchmarks of that year, even after corresponding benchmarks have maybe been changed.

To get started on this, I propose to add a new field `release_timestamp` to [`BenchmarkInstance`](https://github.com/brain-score/brain-score.web/blob/afc5c1a1bf512d8a1f88d4b1c8461001bd13cd42/benchmarks/models.py#L128) that reflects when a particular benchmark (version) was released.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in benchmarks/models.py at BenchmarkInstance and read how its existing fields are defined and persisted. Add the release_timestamp field so each benchmark version can retain its release time; done means the model stores that timestamp for later historical snapshots.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, database
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.