prompt-toolkit / prompt-toolkit/python-prompt-toolkit

`ValueError: I/O operation on closed file` while embedding an IPython within a subprocess

Open
#1,662 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10.6k
Forks
815
PR merge metrics
No merged PRs in 30d

Description

Hello,

I'm having problems embedding an IPython console inside a subprocess when using prompt-toolkit.
I am developing an application using a TUI based on pt. For the subprocesses I am using concurrent.futures.ProcessPoolExecutor.

It seems that, for some reason, the subprocesses close sys.stdin and I can't embed an IPython in them. It only works on the parent process. The following snippet is a minimal reproducible example. It creates a dummy pt Application, spawns a subprocess that tries to execute IPython.embed(). I left the call to stdin_from_terminal() since it is needed when not using the TUI.

$ python -V
Python 3.10.4
$ pip freeze | grep prompt-toolkit
prompt-toolkit==3.0.29
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from prompt_toolkit.layout.dimension import LayoutDimension
from prompt_toolkit.layout.containers import HSplit, Window
from prompt_toolkit.application import Application
from prompt_toolkit.layout.layout import Layout

from concurrent.futures import ProcessPoolExecutor, wait
import contextlib
import threading
import IPython
import sys
import os

# --------------------------------------- #

TTYNAMES = {"posix": "/dev/tty", "nt": "con"}

@contextlib.contextmanager
def stdin_from_terminal():
    try:
        ttyname = TTYNAMES[os.name]
    except KeyError:
        raise OSError(f"{os.name} does not support manually reading from the terminal")
    with open(ttyname) as tty:
        sys.stdin, oldstdin = tty, sys.stdin
        try:
            yield
        finally:
            sys.stdin = oldstdin

# --------------------------------------- #

def worker_fcn():
    print(f"Worker process => PID: {os.getpid()}")
    with stdin_from_terminal():
        IPython.embed()

def init_fcn(args, kwargs):
    pass

def main():

    app = Application(
        layout=Layout(HSplit(
            children = [Window(always_hide_cursor=True)],
            height=LayoutDimension.exact(0),
        )),
        mouse_support=False,
        full_screen=False,
    )
    # app.run()
    tui_thread = threading.Thread(
        target=app.run,
        daemon=True,
        kwargs={'in_thread': False},
    )
    tui_thread.start()

    print(f"Parent process => PID: {os.getpid()}")
    executor = ProcessPoolExecutor(
        max_workers=1,
        initializer=init_fcn,
        initargs=([], {})
    )

    job = executor.submit(worker_fcn)
    ex = job.exception()
    if ex is not None:
        print(f"Exception catched: {ex}")

    executor.shutdown(wait=True)
    app.exit()
    tui_thread.join()

    return 0

if __name__ == '__main__':
    main()

$ python ipython-subprocess.py
Parent process => PID: 12161
Worker process => PID: 12162
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: "It works as expected"
Out[1]: 'It works as expected'

In [2]: exit

$ python ipython-subprocess.py TUI
Parent process => PID: 12168
Worker process => PID: 12170
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.2.0 -- An enhanced Interactive Python. Type '?' for help.

Exception catched: I/O operation on closed file

Is it a Prompt Toolkit error? Is there an option I need to specify that I missed?
Thank you very much in advance.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied minimal example in both modes and compare the paths through Application.run, stdin_from_terminal, and worker_fcn around IPython.embed(). Determine whether prompt-toolkit closes or replaces stdin in the subprocess, then verify that the supported behavior is documented or that the failing case no longer raises the reported ValueError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.