pytest-dev / pytest-dev/pytest-xdist

Random crashes on PyPy 3

Open
#622 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I'm seeing random crashes when running my tests on PyPy 3.6 and 3.7 (7.3.3 on Ubuntu and macOS)

full log 
platform linux -- Python 3.7.9[pypy-7.3.3-beta], pytest-6.1.2, py-1.10.0, pluggy-0.13.1
cachedir: .tox/pypy3/.pytest_cache
rootdir: /home/runner/work/rinohtype/rinohtype, configfile: setup.cfg
plugins: xdist-2.2.0, forked-1.3.0, cov-2.11.1
gw0 I / gw1 I
gw0 [110] / gw1 [110]

............................................................................................WARNING: The config value `latex_documents' has type `NoneType', defaults to `list'.
..................INTERNALERROR> def worker_internal_error(self, node, formatted_error):
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False, formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/_pytest/main.py", line 257, in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/_pytest/main.py", line 313, in _main
INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR> E                 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR> E                 gen.send(outcome)
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pytest_cov/plugin.py", line 271, in pytest_runtestloop
INTERNALERROR> E                 self.cov_controller.finish()
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR> E                 return meth(self, *args, **kwargs)
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pytest_cov/engine.py", line 373, in finish
INTERNALERROR> E                 self.cov.save()
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/coverage/control.py", line 659, in save
INTERNALERROR> E                 data = self.get_data()
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/coverage/control.py", line 717, in get_data
INTERNALERROR> E                 if self._collector and self._collector.flush_data():
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/coverage/collector.py", line 440, in flush_data
INTERNALERROR> E                 self.covdata.add_arcs(self.mapped_file_dict(self.data))
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/coverage/sqldata.py", line 481, in add_arcs
INTERNALERROR> E                 data,
INTERNALERROR> E               File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/coverage/sqldata.py", line 1106, in executemany
INTERNALERROR> E                 return self.con.executemany(sql, data)
INTERNALERROR> E               File "/opt/hostedtoolcache/PyPy/3.7.9/x64/lib_pypy/_sqlite3.py", line 424, in executemany
INTERNALERROR> E                 return cur.executemany(*args)
INTERNALERROR> E               File "/opt/hostedtoolcache/PyPy/3.7.9/x64/lib_pypy/_sqlite3.py", line 821, in wrapper
INTERNALERROR> E                 return func(self, *args, **kwargs)
INTERNALERROR> E               File "/opt/hostedtoolcache/PyPy/3.7.9/x64/lib_pypy/_sqlite3.py", line 986, in executemany
INTERNALERROR> E                 return self.__execute(True, sql, many_params)
INTERNALERROR> E               File "/opt/hostedtoolcache/PyPy/3.7.9/x64/lib_pypy/_sqlite3.py", line 937, in __execute
INTERNALERROR> E                 self.__statement._set_params(params)
INTERNALERROR> E               File "/opt/hostedtoolcache/PyPy/3.7.9/x64/lib_pypy/_sqlite3.py", line 1228, in _set_params
INTERNALERROR> E                 "probably unsupported type." % i)
INTERNALERROR> E             _sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> .tox/pypy3/site-packages/xdist/dsession.py:187: AssertionError
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/_pytest/main.py", line 257, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/_pytest/main.py", line 313, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/home/runner/work/rinohtype/rinohtype/.tox/pypy3/site-packages/xdist/dsession.py", line 175, in worker_workerfinished
INTERNALERROR>     self._active_nodes.remove(node)
INTERNALERROR> KeyError: <WorkerController gw0>

Of course, this could also be a bug in PyPy...

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked test runs and the PyPy log, tracing the failure from pytest-xdist's .tox/pypy3/site-packages/xdist/dsession.py through coverage's SQLite calls and lib_pypy/_sqlite3.py. Determine whether the crash is in xdist, coverage, SQLite handling, or PyPy, then verify that the relevant PyPy test run completes without the internal error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.