brain-score / brain-score/vision
Parallel test execution causes race conditions in the shared test database
- Dominant language
- Python
- Stars
- 193
- Forks
- 105
- Avg merge
- 10h 48m
- Merged PRs (30d)
- 10
Description
Current endpoint unit tests and end-to-end integration tests all run on a single database. Before and after each test, the database is emptied to create isolated conditions for testing. An example of this is [test_endpoints.py](https://github.com/brain-score/brain-score/blob/integrate_core/tests/test_submission/test_endpoints.py).
However, when multiple tests that access the database run in parallel (e.g. tests running on Travis at the same time as tests running on Jenkins), race conditions occur where the database is emptied while another test is using it, or the database is modified by both tests simultaneously, causing both of them to fail. Currently, this is mitigated by re-running the jobs until no collision happens.
Future iterations could address this issue by potentially using a local database instead of the same remote database or by implementing DB-locking on the server side by making each connection to the test db (and all its transactions) an atomic operation.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.