_test_multiprocessing._kill_process() uses a fixed 10 s alarm, failing on slow builds
Personne n'a encore pris cette issue.
- Langage dominant
- Python
- Étoiles
- 77.2k
- Forks
- 35.9k
- Métriques de merge des PR
- Métriques de PR en attente
Description
Bug report
Bug description:
_test_multiprocessing._kill_process() guards the join() of the killed child with a SIGALRM fixed at 10 seconds:
if hasattr(signal, 'alarm'):
# On the Gentoo buildbot waitpid() often seems to block forever.
# We use alarm() to interrupt it if it blocks for too long.
def handler(*args):
raise RuntimeError('join took too long: %s' % p)
old_handler = signal.signal(signal.SIGALRM, handler)
try:
signal.alarm(10)
self.assertEqual(join(), None)
The alarm is meant to turn a blocked waitpid() into a readable error instead of a hang. But the value has been a literal 10 since it was added in 2013 (cc5c728513a), so on a build slow enough that reaping the child legitimately takes longer than ten seconds, it fires on a healthy run and fails the test.
That happened on the "Sanitizers / UBSan" job of 57594aae5e: https://github.com/python/cpython/actions/runs/34016012199/job/101439811595
ERROR: test_interrupt (test.test_multiprocessing_fork.test_processes.WithProcessesTestProcess.test_interrupt)
File ".../Lib/test/_test_multiprocessing.py", line 651, in test_interrupt
exitcode = self._kill_process(multiprocessing.Process.interrupt)
File ".../Lib/test/_test_multiprocessing.py", line 632, in _kill_process
self.assertEqual(join(), None)
File ".../Lib/multiprocessing/popen_fork.py", line 28, in poll
pid, sts = os.waitpid(self.pid, flag)
File ".../Lib/test/_test_multiprocessing.py", line 628, in handler
raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-162' pid=18960 parent=17748 started daemon>
The traceback shows the alarm interrupting os.waitpid(), which is the normal path, not a hang.
test.support already provides timeouts for this, and regrtest scales them for slow builds (Lib/test/libregrtest/setup.py raises SHORT_TIMEOUT and LONG_TIMEOUT from --timeout). LONG_TIMEOUT is documented for exactly this use:
# Timeout in seconds to detect when a test hangs.
#
# It is long enough to reduce the risk of test failure on the slowest Python
# buildbots. It should not be used to mark a test as failed if the test takes
# "too long".
and the comment on SHORT_TIMEOUT says "If a test using SHORT_TIMEOUT starts to fail randomly on slow buildbots, use LONG_TIMEOUT instead". The same _kill_process() already uses support.SHORT_TIMEOUT a few lines above, for the event wait.
Reproducer
The alarm fires whenever the child takes longer than ten seconds to be reaped. Replicating the alarm block with a child that is slow to exit:
import multiprocessing, signal, sys, time
def child(event, delay):
def slow_exit(*args):
time.sleep(delay) # a loaded machine: the child is slow to die
sys.exit(0)
signal.signal(signal.SIGINT, slow_exit)
event.set()
time.sleep(100)
if __name__ == '__main__':
alarm_secs, delay = int(sys.argv[1]), float(sys.argv[2])
event = multiprocessing.Event()
p = multiprocessing.Process(target=child, args=(event, delay))
p.daemon = True
p.start()
event.wait(30)
p.interrupt()
def handler(*args):
raise RuntimeError('join took too long: %s' % p)
signal.signal(signal.SIGALRM, handler)
try:
signal.alarm(alarm_secs)
p.join()
print("join returned, exitcode", p.exitcode)
except RuntimeError as exc:
print("RuntimeError:", exc)
finally:
signal.alarm(0)
$ ./python mp_alarm.py 10 12
RuntimeError: join took too long: <Process name='Process-1' pid=13882 parent=13880 started daemon>
$ ./python mp_alarm.py 300 12
join returned, exitcode 0
The four tests that go through _kill_process() (test_interrupt, test_interrupt_no_handler, test_terminate, test_kill) are affected, in every start-method variant.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux (CI), macOS (reproducer)
Linked PRs
- gh-157185
Guide de contribution
Ouvrir le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Piste de recherche
Commencez dans Lib/test/_test_multiprocessing.py à _kill_process(), puis comparez sa gestion des alarmes avec les définitions et la mise à l’échelle des timeouts de support dans Lib/test/libregrtest/setup.py. Vérifiez les quatre tests concernés avec chaque méthode de démarrage ; le travail est terminé lorsque la récupération d’un processus enfant lent mais sain ne déclenche plus le timeout, tandis que les joins réellement bloqués échouent toujours clairement.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- testing-qa
- Type d'issue
- Bug
- Difficulté
- 2/5
- Temps estimé
- 1-3 heures
- Activité
- À l'abandon
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 35/100