python-trio / python-trio/trio

Using one process's output as another's input creates challenges with non-blocking status

Open
#1,707 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

subprocesses
Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

When launching piped subprocesses, trio provides the stdin of the 2nd process (at least, I have not checked if others are concerned) as O_NONBLOCK while the shell or Python subprocess.Popen() do not (on a Linux platform).

See attachment pipe_nonblock.zip for the scripts used.

$ echo foo | ./desc_stdin.py
stdin F_GETFL 0
$ ./python_launch_pipe.py 
p2 stderr: data=b'stdin F_GETFL 0\n'
main end
$ ./trio_launch_pipe.py 
p2 stderr: data=b'stdin F_GETFL 2048\n'  <-- Here O_NONBLOCK is set
main end

I was expecting that by default, for a new process, all IOs were blocking. As a workaround, I will have to explicitly clear the O_NONBLOCK flag in my spawned exe, but I believe it can create problems with a variety of applications (that are expecting blocking IO by default).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the attached pipe_nonblock.zip scripts, especially trio_launch_pipe.py and python_launch_pipe.py, and reproduce the Linux difference in stdin flags. Trace how Trio launches the piped second process and determine whether its stdin should remain blocking by default. Done means spawned processes behave like the shell and subprocess.Popen without requiring the child to clear O_NONBLOCK.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.