ipython / ipython/ipykernel

Verbose subprocess and c-library hangs with IPyKernel 6.x.x

Offen
#847 1 Kommentar 2 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
734
Forks
412
Ø Merge
1 T. 5 Std.
Gemergte PRs (30 T.)
8

Beschreibung

Since `IPyKernel` 6.0.0, in a notebook that call some functions working with some very verbose subprocess and C/C++ library ends up hanging indefinitely.

This does not happens with `IPyKernel` 5.5.6, but can be circumvent, in 6.3.0 and above, by setting `c.IPKernelApp.capture_fd_output = False` in the config file.

From what I am guessing, this is a bug in the thread that handle the capture of `stdout` and `stderr`that was introduced in 6.0.0.

While playing around with the configuration of the kernel and a notebook, it seems that the thread in IPyKernel which read `stdout` and `stderr` start reading one of them, but all output goes to the other at this moment. Since reading from `stdout` or `stderr` is a blocking operation, the read operation never stop and the other output ends up filling its buffer, at this stage, no one can write to the output, hence the kernel completely hangs.

For me this is probably it since, when we kill the kernel (from the notebook interface), a lot of output suddenly appeared, before having the stack trace for the `KeyboardInterrupt` exception. Just like if one of the output wasn't read for some time, but got flushed when interrupting the kernel.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start by reproducing the hang with IPyKernel 6.x.x using a very verbose subprocess or C/C++ library, comparing capture_fd_output=True with False. Investigate the stdout and stderr capture thread described in the issue; done means the kernel remains responsive and captures output without requiring the workaround.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
jupyter-notebook, python
Bereich
tooling
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
32/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.