brain-score / brain-score/vision

More information about current benchmarks running

Open
#270 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
105
Avg merge
10h 48m
Merged PRs (30d)
10

Description

It would help keeping track of the state of benchmarks being evaluated if we could access which benchmarks/models are currently queued to be scored, and currently being ran. This could be done either by displaying that information in the private area of the website, or perhaps easier to implement, by sending a confirmation email with every benchmark/model request submission.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing benchmark/model request submission and the private area of the website to find where queued and running evaluations are represented. Confirm with maintainers whether the intended outcome is status visibility or a confirmation email, then verify that the chosen path reports submitted benchmark/model requests clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.