pybind / pybind/pybind11

[BUG]: nox -s tests -- -DCMAKE_BUILD_TYPE=Debug -DDOWNLOAD_EIGEN=OFF failed about pybind11@master on centos8_aarch64

Open
#4,441 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

master

Problem description
[root@bigdata spack-src]# nox -s tests -- -DCMAKE_BUILD_TYPE=Debug -DDOWNLOAD_EIGEN=OFF
……
____________________________________________________________________________ test_run_in_process_direct[_intentional_deadlock] ____________________________________________________________________________

test_fn = <function _intentional_deadlock at 0xffff79cf59d0>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_direct(test_fn):
        """Makes sure there is no GIL deadlock when using processes.

        This test is for completion, but it was never an issue.
        """
>       assert _run_in_process(test_fn) == 0

test_fn    = <function _intentional_deadlock at 0xffff79cf59d0>

../../../../tests/test_gil_scoped.py:244:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../../tests/test_gil_scoped.py:161: in _run_in_process
    process.start()
        args       = ()
        kwargs     = {}
        process    = <Process name='Process-72' parent=3362301 initial daemon>
        t_start    = 1672728059.3542063
        target     = <function _intentional_deadlock at 0xffff79cf59d0>
        test_fn    = <function _intentional_deadlock at 0xffff79cf59d0>
        timeout    = 0.1
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
        self       = <Process name='Process-72' parent=3362301 initial daemon>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/context.py:224: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
        process_obj = <Process name='Process-72' parent=3362301 initial daemon>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/context.py:291: in _Popen
    return Popen(process_obj)
        Popen      = <class 'multiprocessing.popen_forkserver.Popen'>
        process_obj = <Process name='Process-72' parent=3362301 initial daemon>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
        __class__  = <class 'multiprocessing.popen_forkserver.Popen'>
        process_obj = <Process name='Process-72' parent=3362301 initial daemon>
        self       = <multiprocessing.popen_forkserver.Popen object at 0xffff79213eb0>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/popen_fork.py:19: in __init__
    self._launch(process_obj)
        process_obj = <Process name='Process-72' parent=3362301 initial daemon>
        self       = <multiprocessing.popen_forkserver.Popen object at 0xffff79213eb0>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/popen_forkserver.py:51: in _launch
    self.sentinel, w = forkserver.connect_to_new_process(self._fds)
        buf        = <_io.BytesIO object at 0xffffa0695220>
        prep_data  = {'authkey': b'\x84\x94\x02$\xf7\x99\xaf<:tl\xccor%\t\xf1_&\xc5\xf7W\xe0\xab\xcf5\x82C>\xc1\x8a\xfd', 'dir': '/home/sta...r5rgfkal5y2/spack-src/.nox/tests-3-8/tmp/tests', 'init_main_from_name': 'pytest.__main__', 'log_to_stderr': False, ...}
        process_obj = <Process name='Process-72' parent=3362301 initial daemon>
        self       = <multiprocessing.popen_forkserver.Popen object at 0xffff79213eb0>
/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/forkserver.py:84: in connect_to_new_process
    self.ensure_running()
        fds        = []
        self       = <multiprocessing.forkserver.ForkServer object at 0xffff7995fd60>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <multiprocessing.forkserver.ForkServer object at 0xffff7995fd60>

    def ensure_running(self):
        '''Make sure that a fork server is running.

        This can be called from any process.  Note that usually a child
        process will just reuse the forkserver started by its parent, so
        ensure_running() will do nothing.
        '''
        with self._lock:
            resource_tracker.ensure_running()
            if self._forkserver_pid is not None:
                # forkserver was launched before, is it still running?
                pid, status = os.waitpid(self._forkserver_pid, os.WNOHANG)
                if not pid:
                    # still alive
                    return
                # dead, launch it again
                os.close(self._forkserver_alive_fd)
                self._forkserver_address = None
                self._forkserver_alive_fd = None
                self._forkserver_pid = None

            cmd = ('from multiprocessing.forkserver import main; ' +
                   'main(%d, %d, %r, **%r)')

            if self._preload_modules:
                desired_keys = {'main_path', 'sys_path'}
                data = spawn.get_preparation_data('ignore')
                data = {x: y for x, y in data.items() if x in desired_keys}
            else:
                data = {}

            with socket.socket(socket.AF_UNIX) as listener:
                address = connection.arbitrary_address('AF_UNIX')
>               listener.bind(address)
E               OSError: AF_UNIX path too long

address    = '/home/stage/root/spack-stage-py-pybind11-master-buhdtf3eegctgi7bjpxzrr5rgfkal5y2/spack-src/.nox/tests-3-8/tmp/pymp-2y6zv958/listener-xwlet19c'
cmd        = 'from multiprocessing.forkserver import main; main(%d, %d, %r, **%r)'
data       = {'sys_path': ['/home/stage/root/spack-stage-py-pybind11-master-buhdtf3eegctgi7bjpxzrr5rgfkal5y2/spack-src/tests', '/ho.../spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python38.zip', ...]}
desired_keys = {'main_path', 'sys_path'}
listener   = <socket.socket [closed] fd=-1, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
self       = <multiprocessing.forkserver.ForkServer object at 0xffff7995fd60>

/home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/python-3.8.8-m3k3pvne56yo7wyd7a3bwf7mugupszik/lib/python3.8/multiprocessing/forkserver.py:138: OSError
========================================================================================= short test summary info =========================================================================================
SKIPPED [1] ../../../../tests/test_builtin_casters.py:145: no <string_view>
SKIPPED [1] ../../../../tests/test_callbacks.py:203: Current PYBIND11_INTERNALS_VERSION too low
SKIPPED [1] ../../../../tests/test_copy_move.py:77: no <optional>
SKIPPED [1] ../../../../tests/test_sequences_and_iterators.py:13: no <optional>
SKIPPED [1] ../../../../tests/test_stl.py:111: no <optional>
SKIPPED [1] ../../../../tests/test_stl.py:143: no <experimental/optional>
SKIPPED [1] ../../../../tests/test_stl.py:175: no <boost/optional>
SKIPPED [1] ../../../../tests/test_stl.py:234: no <filesystem>
SKIPPED [1] ../../../../tests/test_stl.py:253: no <variant>
SKIPPED [1] ../../../../tests/test_stl.py:270: no std::monostate
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_callback_py_obj] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_callback_std_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_callback_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_callback_pure_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_inner_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_inner_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_inner_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_inner_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_nested_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_nested_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_nested_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_cross_module_gil_nested_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_release_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_nested_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[test_multi_acquire_release_cross_module] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_one_thread[_intentional_deadlock] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_callback_py_obj] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_callback_std_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_callback_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_callback_pure_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_inner_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_inner_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_inner_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_inner_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_nested_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_nested_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_nested_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_nested_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_release_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_nested_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_multi_acquire_release_cross_module] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[_intentional_deadlock] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_callback_py_obj] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_callback_std_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_callback_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_callback_pure_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_nested_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_nested_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_nested_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_nested_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_release_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_nested_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_multi_acquire_release_cross_module] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[_intentional_deadlock] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_callback_py_obj] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_callback_std_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_callback_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_callback_pure_virtual_func] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_inner_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_inner_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_inner_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_inner_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_nested_custom_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_nested_custom_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_nested_pybind11_released] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_cross_module_gil_nested_pybind11_acquired] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_release_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_nested_acquire] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[test_multi_acquire_release_cross_module] - OSError: AF_UNIX path too long
FAILED ../../../../tests/test_gil_scoped.py::test_run_in_process_direct[_intentional_deadlock] - OSError: AF_UNIX path too long
=============================================================================== 72 failed, 711 passed, 10 skipped in 28.18s ===============================================================================
gmake[3]: *** [tests/CMakeFiles/pytest.dir/build.make:74: tests/CMakeFiles/pytest] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:385: tests/CMakeFiles/pytest.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:450: tests/CMakeFiles/check.dir/rule] Error 2
gmake: *** [Makefile:274: check] Error 2
nox > Command cmake --build .nox/tests-3-8/tmp --config=Release --target check failed with exit code 2
nox > Session tests-3.8 failed.
nox > Running session tests-3.9
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-3.9 skipped: Python interpreter 3.9 not found.
nox > Running session tests-3.10
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-3.10 skipped: Python interpreter 3.10 not found.
nox > Running session tests-3.11
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-3.11 skipped: Python interpreter 3.11 not found.
nox > Running session tests-pypy3.7
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-pypy3.7 skipped: Python interpreter pypy3.7 not found.
nox > Running session tests-pypy3.8
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-pypy3.8 skipped: Python interpreter pypy3.8 not found.
nox > Running session tests-pypy3.9
nox > Missing interpreters will error by default on CI systems.
nox > Session tests-pypy3.9 skipped: Python interpreter pypy3.9 not found.
nox > Ran multiple sessions:
nox > * tests-3.8: failed
nox > * tests-3.9: skipped
nox > * tests-3.10: skipped
nox > * tests-3.11: skipped
nox > * tests-pypy3.7: skipped
nox > * tests-pypy3.8: skipped
nox > * tests-pypy3.9: skipped
Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running nox -s tests -- -DCMAKE_BUILD_TYPE=Debug -DDOWNLOAD_EIGEN=OFF on the reported CentOS 8 AArch64 setup. Inspect tests/test_gil_scoped.py, especially _run_in_process and the failing process tests, alongside the AF_UNIX path too long traceback. Done means the affected tests complete successfully without this socket-path failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.