pytest-dev / pytest-dev/pytest-xdist

Displaying file names when run parallel

Open
#450 10 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Currently when tests are run on multiple cores the name of the files are not displayed any more. It would be rather useful to be able to still receive the same information even if it means one need to wait until all the end of the test run.

Without using numprocesses:

collected 114 items / 28 skipped / 86 selected                                                                                                                    

astropy/io/misc/yaml.py .                                                                                                                                   [  0%]
astropy/io/misc/tests/test_hdf5.py ........................................................ssss...........ss.                                               [ 65%]
astropy/io/misc/tests/test_pandas.py .....                                                                                                                  [ 70%]
astropy/io/misc/tests/test_pickle_helpers.py ....                                                                                                           [ 73%]
astropy/io/misc/tests/test_yaml.py ..............................                                                                                           [100%]

With numprocesses:

rootdir: /Users/bsipocz/munka/devel/astropy, inifile: setup.cfg
plugins: xdist-1.29.0, forked-1.0.2, arraydiff-0.3, cov-2.6.0, xonsh-0.8.3, dependency-0.4.0, doctestplus-0.3.0, requests-mock-1.6.0, openfiles-0.4.0.dev0, remotedata-0.3.2.dev0
gw0 C / gw1 C/usr/local/lib/python3.7/site-packages/pytest_doctestplus/plugin.py:10: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/usr/local/lib/python3.7/site-packages/pytest_doctestplus/plugin.py:10: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
gw0 [114] / gw1 [114]1 skipped                                                                                                                                    
...................................................ssss.............s.s...........................................                                          [100%]

Also, having the info of the total number of collected and skipped tests is useful, too, and it's not printed in the latter case either.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the two displayed runs with pytest-xdist and parallel workers, comparing serial output with the current worker output. Trace the xdist reporting and aggregation entry points, then verify that parallel runs retain file names plus collected and skipped totals without breaking the existing progress display.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, distributed-systems, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.