brain-score / brain-score/vision
Handle multi-plugin submissions elegantly
Open
- Dominant language
- Python
- Stars
- 193
- Forks
- 105
- Avg merge
- 10h 48m
- Merged PRs (30d)
- 10
Description
Right now we cap web submissions at 5 plugins to prevent tests from timing out (see [vision issue #91](https://github.com/brain-score/core/issues/91)). This is a burden on users, and additionally will still occasionally time out.
Long term, we would like to handle larger submissions with some considerations for what sorts of bulk submissions we allow.
* [ ] Consider batching large submissions into either
* multiple PRs
* separate jobs on Travis
* [ ] Consider whether we want to add checking to only allow certain kinds of bulk submission
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.