python / python/cpython

matchfile Worker bug argument list too long when using matchfile to run many tests

Open
#139,207 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tests type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

In https://github.com/python/cpython/issues/138313#issuecomment-3264096281, I'm trying to select different subsets of test_importlib to determine which ones are implicated in the failures. Unfortunately, it seems that when extracting the list of tests and then passing that to --matchfile the execution fails with:

+ ./python -u -W error -bb -E -m test --fast-ci --junit-xml test-results.xml test_importlib --matchfile cases.txt --dont-add-python-opts
Using random seed: 893560730
0:00:00 load avg: 0.54 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)
Warning -- regrtest worker thread failed: Traceback (most recent call last):
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 403, in run
Warning --     mp_result = self._runtest(test_name)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 363, in _runtest
Warning --     retcode, tmp_files = self.run_tmp_files(worker_runtests,
Warning --                          ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
Warning --                                             stdout_file.fileno())
Warning --                                             ^^^^^^^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 299, in run_tmp_files
Warning --     retcode = self._run_process(worker_runtests,
Warning --                                 stdout_fd, tmp_dir)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 180, in _run_process
Warning --     popen = create_worker_process(runtests, output_fd, tmp_dir)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/worker.py", line 73, in create_worker_process
Warning --     return subprocess.Popen(cmd, **kwargs)
Warning --            ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/subprocess.py", line 1037, in __init__
Warning --     self._execute_child(args, executable, preexec_fn, close_fds,
Warning --     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Warning --                         pass_fds, cwd, env,
Warning --                         ^^^^^^^^^^^^^^^^^^^
Warning --     ...<5 lines>...
Warning --                         gid, gids, uid, umask,
Warning --                         ^^^^^^^^^^^^^^^^^^^^^^
Warning --                         start_new_session, process_group)
Warning --                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/subprocess.py", line 1969, in _execute_child
Warning --     raise child_exception_type(errno_num, err_msg, err_filename)
Warning -- OSError: [Errno 7] Argument list too long: '/home/stan/cpython/python'

== Tests result: FAILURE, WORKER BUG ==

It seems there's a limit to the number of lines that can be passed to a --matchfile.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the command with a large cases.txt, then inspect Lib/test/libregrtest/run_workers.py and Lib/test/libregrtest/worker.py around worker process creation. Trace how the --matchfile selection reaches subprocess.Popen and define completion as running the large selection without OSError: [Errno 7] Argument list too long.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.