pytest-dev / pytest-dev/pytest-xdist

'module' object has no attribute 'GtkWarning'

Open
#404 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Hello. First of all I would like to thank everyone involved in making and maintaining pytest and xdist, superb job. :) 👍

Now the situation that is manifesting on my windows7 machine.
I'm trying to assemble CI infrastructure using jenkins for running gui tests via pytest and xdist.
I decided to use xdist because python process sometimes crashes and I want to continue running the remaining number of tests. I've managed to reproduce situation bellow also in the console (cmd.exe) so I don't think this is jenkins related.

Bellow we can see one of the commands I'm using to run pytest, I've moved some args onto a new line so it is easier on the eyes.

C:\Users\user\.jenkins\workspace\Rmaster_GUI>(D: &&
D:/python_folder/python -m pytest --disable-pytest-warnings --showlocals -n1
--cov=D:/installation_folder/python/client/project_name
--cov-report xml:C:\Users\user\.jenkins\workspace\Rmaster_GUI/coverage-reports/coverage.xml
--junit-xml=C:\Users\user\.jenkins\workspace\Rmaster_GUI/test-reports/results.xml
-v
C:\Users\user\.jenkins\workspace\Rmaster_GUI/test/project_name_test )

17:54:47 ============================= test session starts =============================
17:54:47 platform win32 -- Python 2.7.10, pytest-4.1.1, py-1.5.4, pluggy-0.8.1 -- D:\python_folder\python.exe
17:54:47 cachedir: .pytest_cache
17:54:47 rootdir: D:\, inifile:
17:54:47 plugins: xdist-1.26.0, forked-1.0.1, cov-2.5.1
17:54:47 gw0 I
17:54:47 
[gw0] win32 Python 2.7.10 cwd: D:\
17:54:48 
[gw0] Python 2.7.10 (default, Nov 29 2018, 13:23:51) [MSC v.1900 64 bit (AMD64)]
17:55:45 gw0 [1096]
17:55:45 
17:55:45 scheduling tests via LoadScheduling

## After some time and a couple tests ...

17:59:22 [gw0] [  1%] PASSED integration/gui/test_some_component.py::TestSomeComponentEditor::test_component !!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 
17:59:22 INTERNALERROR> Traceback (most recent call last):
17:59:22 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
17:59:22 INTERNALERROR>     kwargs["warning_message_data"]
17:59:22 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
17:59:22 INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
17:59:22 INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 [gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 [gw0] [  1%] FAILED integration/gui/test_some_component.py::TestSomeComponentEditor::test_component
17:59:22 Replacing crashed worker gw0
17:59:22 
[gw1] win32 Python 2.7.10 cwd: D:\
17:59:22 Coverage.py warning: No data was collected. (no-data-collected)
17:59:37 INTERNALERROR> Traceback (most recent call last):
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 203, in wrap_session
17:59:37 INTERNALERROR>     session.exitstatus = doit(config, session) or 0
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 243, in _main
17:59:37 INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\hooks.py", line 284, in __call__
17:59:37 INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 68, in _hookexec
17:59:37 INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 62, in <lambda>
17:59:37 INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 203, in _multicall
17:59:37 INTERNALERROR>     gen.send(outcome)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 81, in get_result
17:59:37 INTERNALERROR>     _reraise(*ex)  # noqa
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 187, in _multicall
17:59:37 INTERNALERROR>     res = hook_impl.function(*args)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 115, in pytest_runtestloop
17:59:37 INTERNALERROR>     self.loop_once()
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 138, in loop_once
17:59:37 INTERNALERROR>     call(**kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 258, in worker_runtest_protocol_complete
17:59:37 INTERNALERROR>     self.sched.mark_test_complete(node, item_index, duration)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\scheduler\load.py", line 151, in mark_test_complete
17:59:37 INTERNALERROR>     self.node2pending[node].remove(item_index)
17:59:37 INTERNALERROR> KeyError: <WorkerController gw0>
17:59:47 
17:59:47 ============= 18 failed, 1 passed, 312 warnings in 298.89 seconds =============

Above code was executed on jenkins and the one bellow in the console on the same computer.

D:\>my_python -m pytest --disable-pytest-warnings -v -l -n1 C:\Users\user\.jenkins\workspace\master_GUI\test\client_test\integration\gui
============================= test session starts =============================
platform win32 -- Python 2.7.10, pytest-4.1.1, py-1.5.4, pluggy-0.8.1 -- D:\python_folder\python.exe
cachedir: .pytest_cache
rootdir: D:\, inifile:
plugins: xdist-1.26.0, forked-1.0.1, cov-2.5.1
[gw0] win32 Python 2.7.10 cwd: D:\
[gw0] Python 2.7.10 (default, Nov 29 2018, 13:23:51) [MSC v.1900 64 bit (AMD64)]
gw0 [27]
scheduling tests via LoadScheduling

test_component.py::TestComponentEditor::test_component
[gw0] [ 37%] PASSED test_some_other_component.py::test_some_other_component !!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: Attr
ibuteError: 'module' object has no attribute 'GtkWarning'

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
INTERNALERROR>     kwargs["warning_message_data"]
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
[gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
[gw0] [ 37%] FAILED test_some_other_component.py::test_some_other_component
Replacing crashed worker gw0
!!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
INTERNALERROR>     kwargs["warning_message_data"]
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
[gw1] win32 Python 2.7.10 cwd: D:\
gw1 C[gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
Replacing crashed worker gw0
Exception KeyError: (1,) in <function remove at 0x0000000004162208> ignored
[gw2] win32 Python 2.7.10 cwd: D:\
gw2 CINTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 203, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 243, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 81, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 115, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 138, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 208, in worker_errordown
INTERNALERROR>     self._active_nodes.remove(node)
INTERNALERROR> KeyError: <WorkerController gw0>

============ 4 failed, 7 passed, 68736 warnings in 784.27 seconds =============

As far as I can see not all tests are being executed because all nodes crash somewhere along the path and don't restart.
Traceback in jenkins and cmd are equal up to the line 138, in loop_once where after they start to diverge but end up with the same KeyError.

I'm also using pytest and xdist for running non gui tests and they work perfectly ok.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with xdist/workermanage.py at line 436, where warning_message_data is deserialized, then trace the worker lifecycle through xdist/dsession.py line 138. Reproduce the Windows Python 2.7 run with -n1 and the reported GtkWarning failure. Done means warning handling no longer crashes the worker and the remaining tests continue executing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.