brain-score / brain-score/vision

use xarray netcdf storing for all cached results

Open
#260 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
105
Avg merge
10h 48m
Merged PRs (30d)
10

Description

The scores of models on benchmarks or ceiling scores for benchmarks are cached using https://github.com/brain-score/result_caching's `@store` method. `@store` uses pickle to serialize and deserialize objects for disk IO which has issues with forward compatibility. To keep things better compatible, we should use [store_xarray](https://github.com/brain-score/result_caching/blob/94cb58c8b3a0a993aec9423122b62928af5e1812/result_caching/__init__.py#L465)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the benchmark and ceiling-score cache uses of result_caching's @store method. Compare those call sites with the referenced store_xarray implementation, then update the cached-result paths to use it and verify that storing and loading all affected results remains compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.