brain-score / brain-score/vision

Improve efficiency of test_look_at_neural_V1

Open
#998 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
105
Avg merge
10h 48m
Merged PRs (30d)
10

Description

### Issue
[`test_look_at_neural_V1`](https://github.com/brain-score/vision/blob/b84502aad25c443419e59a9579cab82110db2323/brainscore_vision/model_helpers/generic_plugin_tests.py#L45) requires so much memory that it leads to some test jobs being killed in OpenMind (see #996, #950, error below). It's already marked `@pytest.mark.memory_intense`, which stops it from running in Travis, but it appears to be failing even in resource-rich environments.

### Task
Either (a) improve the efficiency of the test or (b) remove it from the test set and replace with a reasonable alternative, if possible.

### Error
``` sh
slurmstepd: error: Detected 1 oom-kill event(s) in StepId=37205392.0.
Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: node107: task 0: Out Of Memory
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with brainscore_vision/model_helpers/generic_plugin_tests.py at test_look_at_neural_V1, then inspect the related failures in #996 and #950. Reproduce the memory-intensive test in a resource-rich environment and determine whether it can be made more efficient; done means the test no longer causes out-of-memory failures, or a reasonable replacement is documented and used.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.