python / python/cpython

_test_multiprocessing._kill_process() uses a fixed 10 s alarm, failing on slow builds

Aperta
#157,184 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

tests topic-multiprocessing type-bug
Lingua principale
Python
Stelle
77.2k
Fork
35.9k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

Bug report

Bug description:

_test_multiprocessing._kill_process() guards the join() of the killed child with a SIGALRM fixed at 10 seconds:

        if hasattr(signal, 'alarm'):
            # On the Gentoo buildbot waitpid() often seems to block forever.
            # We use alarm() to interrupt it if it blocks for too long.
            def handler(*args):
                raise RuntimeError('join took too long: %s' % p)
            old_handler = signal.signal(signal.SIGALRM, handler)
            try:
                signal.alarm(10)
                self.assertEqual(join(), None)

The alarm is meant to turn a blocked waitpid() into a readable error instead of a hang. But the value has been a literal 10 since it was added in 2013 (cc5c728513a), so on a build slow enough that reaping the child legitimately takes longer than ten seconds, it fires on a healthy run and fails the test.

That happened on the "Sanitizers / UBSan" job of 57594aae5e: https://github.com/python/cpython/actions/runs/34016012199/job/101439811595

ERROR: test_interrupt (test.test_multiprocessing_fork.test_processes.WithProcessesTestProcess.test_interrupt)
  File ".../Lib/test/_test_multiprocessing.py", line 651, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File ".../Lib/test/_test_multiprocessing.py", line 632, in _kill_process
    self.assertEqual(join(), None)
  File ".../Lib/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
  File ".../Lib/test/_test_multiprocessing.py", line 628, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-162' pid=18960 parent=17748 started daemon>

The traceback shows the alarm interrupting os.waitpid(), which is the normal path, not a hang.

test.support already provides timeouts for this, and regrtest scales them for slow builds (Lib/test/libregrtest/setup.py raises SHORT_TIMEOUT and LONG_TIMEOUT from --timeout). LONG_TIMEOUT is documented for exactly this use:

# Timeout in seconds to detect when a test hangs.
#
# It is long enough to reduce the risk of test failure on the slowest Python
# buildbots. It should not be used to mark a test as failed if the test takes
# "too long".

and the comment on SHORT_TIMEOUT says "If a test using SHORT_TIMEOUT starts to fail randomly on slow buildbots, use LONG_TIMEOUT instead". The same _kill_process() already uses support.SHORT_TIMEOUT a few lines above, for the event wait.

Reproducer

The alarm fires whenever the child takes longer than ten seconds to be reaped. Replicating the alarm block with a child that is slow to exit:

import multiprocessing, signal, sys, time

def child(event, delay):
    def slow_exit(*args):
        time.sleep(delay)      # a loaded machine: the child is slow to die
        sys.exit(0)
    signal.signal(signal.SIGINT, slow_exit)
    event.set()
    time.sleep(100)

if __name__ == '__main__':
    alarm_secs, delay = int(sys.argv[1]), float(sys.argv[2])
    event = multiprocessing.Event()
    p = multiprocessing.Process(target=child, args=(event, delay))
    p.daemon = True
    p.start()
    event.wait(30)
    p.interrupt()

    def handler(*args):
        raise RuntimeError('join took too long: %s' % p)
    signal.signal(signal.SIGALRM, handler)
    try:
        signal.alarm(alarm_secs)
        p.join()
        print("join returned, exitcode", p.exitcode)
    except RuntimeError as exc:
        print("RuntimeError:", exc)
    finally:
        signal.alarm(0)
$ ./python mp_alarm.py 10 12
RuntimeError: join took too long: <Process name='Process-1' pid=13882 parent=13880 started daemon>
$ ./python mp_alarm.py 300 12
join returned, exitcode 0

The four tests that go through _kill_process() (test_interrupt, test_interrupt_no_handler, test_terminate, test_kill) are affected, in every start-method variant.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux (CI), macOS (reproducer)

Linked PRs
  • gh-157185

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia in Lib/test/_test_multiprocessing.py, in _kill_process(), quindi confronta la gestione degli allarmi con le definizioni e il ridimensionamento dei timeout di support in Lib/test/libregrtest/setup.py. Controlla i quattro test interessati con ciascun metodo di avvio; il lavoro è completato quando il reaping di un processo figlio lento ma sano non attiva più il timeout, mentre i join realmente bloccati continuano a fallire in modo chiaro.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
testing-qa
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Ferma
Chiarezza
Specificata chiaramente
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.