brain-score / brain-score/vision
Improve efficiency of test_look_at_neural_V1
- Dominant language
- Python
- Stars
- 193
- Forks
- 105
- Avg merge
- 10h 48m
- Merged PRs (30d)
- 10
Description
### Issue
[`test_look_at_neural_V1`](https://github.com/brain-score/vision/blob/b84502aad25c443419e59a9579cab82110db2323/brainscore_vision/model_helpers/generic_plugin_tests.py#L45) requires so much memory that it leads to some test jobs being killed in OpenMind (see #996, #950, error below). It's already marked `@pytest.mark.memory_intense`, which stops it from running in Travis, but it appears to be failing even in resource-rich environments.
### Task
Either (a) improve the efficiency of the test or (b) remove it from the test set and replace with a reasonable alternative, if possible.
### Error
``` sh
slurmstepd: error: Detected 1 oom-kill event(s) in StepId=37205392.0.
Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: node107: task 0: Out Of Memory
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with brainscore_vision/model_helpers/generic_plugin_tests.py at test_look_at_neural_V1, then inspect the related failures in #996 and #950. Reproduce the memory-intensive test in a resource-rich environment and determine whether it can be made more efficient; done means the test no longer causes out-of-memory failures, or a reasonable replacement is documented and used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100