brain-score / brain-score/language

add behavioral benchmarks from Huang et al. 2023

Open
#169 0 comments 0 reactions 0 assignees View on GitHub
benchmark-request
Dominant language
Jupyter Notebook
Stars
45
Forks
21
Avg merge
1h 7m
Merged PRs (30d)
1

Description

![image](https://user-images.githubusercontent.com/5308236/233753149-7c9bfdaf-8e51-4b7f-a256-1bdcd06f4236.png)
https://psyarxiv.com/z38u6

8 new behavioral benchmarks that models seem to struggle with. They estimate ceilings, provide a metric to compare models to data, this is great. Should just be implementation without any conceptual decisions.

Data on OSF: https://osf.io/b6rqh/

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.