Deadlock in pytest tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 899
- PR merge metrics
- No merged PRs in 30d
Description
I encountered an issue when building this software with the Nix package manager on NixOS. Our package uses commit 536b6e1414c8a93f0815dbae85d03749daaa5432 because there is no recent release.
nix-build '<nixpkgs>' -A python310Packages.opensfm --check
The build is often stuck at a very specific point:
...
opensfm/test/test_dense.py .. [ 55%]
opensfm/test/test_geo.py ..... [ 57%]
opensfm/test/test_geometry.py .... [ 59%]
opensfm/test/test_io.py ......... [ 62%]
opensfm/test/test_matching.py ...... [ 65%]
opensfm/test/test_multiview.py ......... [ 68%]
opensfm/test/test_pairs_selection.py ......... [ 72%]
opensfm/test/test_reconstruction_alignment.py ........ [ 75%]
opensfm/test/test_reconstruction_incremental.py Terminated
So i have to kill the build process. I waited over 3.5 hours once!
Maybe the last test creates a deadlock?
When i look at the process with strace, it only does sched_yield calls.
I have seen similar issues with other python packages doing pytest tests and only doing futex calls at some point.
This seem to happen especially when multiple python packages are built at the same time.
Downstream issue: https://github.com/NixOS/nixpkgs/issues/217247
Our package definition: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/development/python-modules/opensfm/default.nix
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the Nix check with nix-build '<nixpkgs>' -A python310Packages.opensfm --check, then focus on opensfm/test/test_reconstruction_incremental.py, where the run stops. Compare behavior when Python packages build concurrently and inspect the process with strace. Done means the pytest suite completes reliably instead of hanging at that test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nixos, python
- Domain
- build-system, computer-vision, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100