python / python/cpython

matchfile Worker bug argument list too long when using matchfile to run many tests

Abierto
#139,207 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

tests type-bug
Lenguaje dominante
Python
Estrellas
77.2k
Forks
35.9k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Bug report

Bug description:

In https://github.com/python/cpython/issues/138313#issuecomment-3264096281, I'm trying to select different subsets of test_importlib to determine which ones are implicated in the failures. Unfortunately, it seems that when extracting the list of tests and then passing that to --matchfile the execution fails with:

+ ./python -u -W error -bb -E -m test --fast-ci --junit-xml test-results.xml test_importlib --matchfile cases.txt --dont-add-python-opts
Using random seed: 893560730
0:00:00 load avg: 0.54 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)
Warning -- regrtest worker thread failed: Traceback (most recent call last):
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 403, in run
Warning --     mp_result = self._runtest(test_name)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 363, in _runtest
Warning --     retcode, tmp_files = self.run_tmp_files(worker_runtests,
Warning --                          ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
Warning --                                             stdout_file.fileno())
Warning --                                             ^^^^^^^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 299, in run_tmp_files
Warning --     retcode = self._run_process(worker_runtests,
Warning --                                 stdout_fd, tmp_dir)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/run_workers.py", line 180, in _run_process
Warning --     popen = create_worker_process(runtests, output_fd, tmp_dir)
Warning --   File "/home/stan/cpython/Lib/test/libregrtest/worker.py", line 73, in create_worker_process
Warning --     return subprocess.Popen(cmd, **kwargs)
Warning --            ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/subprocess.py", line 1037, in __init__
Warning --     self._execute_child(args, executable, preexec_fn, close_fds,
Warning --     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Warning --                         pass_fds, cwd, env,
Warning --                         ^^^^^^^^^^^^^^^^^^^
Warning --     ...<5 lines>...
Warning --                         gid, gids, uid, umask,
Warning --                         ^^^^^^^^^^^^^^^^^^^^^^
Warning --                         start_new_session, process_group)
Warning --                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Warning --   File "/home/stan/cpython/Lib/subprocess.py", line 1969, in _execute_child
Warning --     raise child_exception_type(errno_num, err_msg, err_filename)
Warning -- OSError: [Errno 7] Argument list too long: '/home/stan/cpython/python'

== Tests result: FAILURE, WORKER BUG ==

It seems there's a limit to the number of lines that can be passed to a --matchfile.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza reproduciendo el comando con un cases.txt grande y, después, inspecciona Lib/test/libregrestest/run_workers.py y Lib/test/libregrestest/worker.py alrededor de la creación de procesos worker. Rastrea cómo la selección de --matchfile llega a subprocess.Popen y define la finalización como ejecutar la selección grande sin OSError: [Errno 7] Argument list too long.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python
Área
testing-qa
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
45/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.