Last-failed option missing tests when failed tests include a mix stable and random node IDs

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using

  attrs (22.1.0)
  Faker (14.2.0)
  importlib-metadata (4.12.0)
  iniconfig (1.1.1)
  packaging (21.3)
  pip (9.0.1)
  pkg-resources (0.0.0)
  pluggy (1.0.0)
  py (1.11.0)
  pyparsing (3.0.9)
  pytest (7.1.2)
  pytest-faker (2.0.0)
  python-dateutil (2.8.2)
  setuptools (39.0.1)
  six (1.16.0)
  tomli (2.0.1)
  typing-extensions (4.3.0)
  zipp (3.8.1)
  

• pytest and operating system versions
  • Python 3.7.5
  • pytest 7.1.2
  • Ubuntu 18.04.6
• minimal example if possible

1. Start with the given test code.

   import pytest
   from faker import Faker
   fake = Faker()
   
   @pytest.mark.parametrize('a', (fake.pystr(),))
   def test_asd(a):
       assert False
   
   def test_zxc():
       assert False
   

2. Run pytest once

   pytest --tb=no -q
   FF                                                                                             [100%]
   ====================================== short test summary info =======================================
   FAILED test_asd.py::test_asd[fPLeEKOcteuKAoJyUmpz] - assert False
   FAILED test_asd.py::test_zxc - assert False
   2 failed in 0.05s
   

3. Run pytest again with the --last-failed option

   pytest --tb=no -q --lf
   F                                                                                              [100%]
   ====================================== short test summary info =======================================
   FAILED test_asd.py::test_zxc - assert False
   1 failed in 0.05s
   

4. Expect both tests to be rerun - only test_zxc is rerun

Initially we thought this was because test_asd is using a randomized parameterized fixture. Since the node IDs are essentially random it would make sense pytest doesn't reselect them.

But if you remove test_zxc in the example above and run steps 2 and 3, pytest actually works and reruns test_asd and the node IDs are different each time.

pytest --tb=no -q; pytest --tb=no -q --lf
F                                                                                              [100%]
====================================== short test summary info =======================================
FAILED test_asd.py::test_asd[IBWBNcpQQilmccBLPmIB] - assert False
1 failed in 0.05s
F                                                                                              [100%]
====================================== short test summary info =======================================
FAILED test_asd.py::test_asd[AsUfmfLBeTdfakmHXPmm] - assert False
1 failed in 0.05s

Start with pytest's --last-failed implementation and reproduce the report using the parametrized test and ordinary failing test shown in the issue. Trace how failed node IDs are stored and selected when stable and randomized IDs are mixed. Done means a subsequent --last-failed run reruns both failures, with regression coverage for this mixed case.