brain-score / brain-score/vision

Submitting a model with same identifier uses previous score

Open
#258 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
105
Avg merge
10h 48m
Merged PRs (30d)
10

Description

When a model is changed and re-submitted with the same identifier, no new scores will be computed: https://github.com/brain-score/brain-score/blob/67e4d01f78634fd5c38ae0668f37d0e5555746db/brainscore/submission/evaluation.py#L109-L112
Rather, the previously computed scores will be reported back.
We should just re-compute at this point and use the most recent submission in the display.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.