pytest-dev / pytest-dev/pytest-xdist

Different tests were collected between workers

Open
#366 18 comments 10 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs information question
Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

I see there are tickets opened and closed related to the same issue in the past. The tickets were closed with the reason saying the the issue is due to parameterized tests and that the issue is with sets bring unordered vs a sorted list not being unordered.

Here are the links to old tickets
https://github.com/pytest-dev/pytest-xdist/issues/187
https://github.com/pytest-dev/pytest-xdist/issues/149

In my case, my tests are not parameterized at all but still facing the same issue. I also tried having 'pythonhashseed=0' in pytest.ini file with pytest-env installed but that did not help. I feel the issue is not only with parameterized tests. I've python3.6 with latest pytest and pytest-xdist.

___________________________________________________________________________________ ERROR collecting gw23 ___________________________________________________________________________________
Different tests were collected between gw0 and gw23. The difference is:
--- gw0

+++ gw23

@@ -1,84 +0,0 @@

-test.py::test_01_verify_connectivity_to_device_vpn_zone_default_vlan_ip_from_vpn_zone_client
-test.py::test_02_verify_traffic_from_vpn_to_dmz_blackholed_by_default_perimeter_policy
-test.py::test_03_verify_traffic_from_vpn_to_dmz_allowed_by_perimeter_policy
-test.py::test_04_verify_traffic_from_vpn_to_dmz_allowed_by_nml_hit_policy
-test.py::test_05_verify_traffic_from_vpn_to_dmz_blackholed_by_user_defined_perimeter_policy
-test.py::test_06_verify_traffic_from_vpn_to_dmz_blackholed_by_user_defined_nml_hit_policy
-test.py::test_07_verify_traffic_from_vpn_to_dmz_dropped_by_unrechable_action_in_user_defined_perimeter_policy
-test.py::test_08_verify_traffic_from_vpn_to_dmz_dropped_by_unrechable_action_in_nml_hit_policy
-test.py::test_09_verify_traffic_from_vpn_to_dmz_dropped_by_prohibit_action_in_user_defined_perimeter_policy
-test.py::test_10_verify_traffic_from_vpn_to_dmz_dropped_by_prohibit_action_in_user_defined_nml_hit_policy

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing historical issues #187 and #149, then reproduce the collection mismatch from test.py across gw0 and gw23 with the reported non-parameterized tests. Compare the workers' collection inputs and check whether pytest.ini or pythonhashseed changes the result; done means identifying a reproducible cause and documenting or testing the fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.