subprocess.Popen on windows cannot use default handles for stdin/stdout/stderr when some of them are redirected
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
suppose I want to run a subprocess with piped stdin/stdout but let that process print to console using stderr (which is a common use pattern)
currently in Popen there's no way to override stdin=PIPE, stdout=PIPE but let stderr use default handle
if I have a child.py script:
import sys, time
try:
print('console message', file=sys.stderr)
except Exception as e:
open('CONOUT$', 'w').write(str(e))
while True:
time.sleep(1)
and parent.py script:
import subprocess
with subprocess.Popen(('python.exe', 'child.py'), stdin=subprocess.PIPE, stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE):
pass
then it works if parent script is run in a console. But if it is a GUI script (e.g. run via pythonw.exe) then child script attempting to print to stderr gets [Errno 22] Invalid argument
this happens because this line:
https://github.com/python/cpython/blob/067145177975eadd61a0c907d0d177f7b6a5a3de/Lib/subprocess.py#L1398
tries to inherit stderr, but it returns None in a GUI app, so then a fake Pipe is created and ultimately passed to CreateProcess as stderr instead of NULL handle.
Here's a hack which allows me to selectively pipe only stdin and stdout and leave stderr as default:
import subprocess
class HackedSTARTUPINFO(subprocess.STARTUPINFO):
def __setattr__(self, attr, value):
if attr == 'hStdError':
value = None
super().__setattr__(attr, value)
def copy(self):
return self
si = HackedSTARTUPINFO()
with subprocess.Popen(('python.exe', 'child.py'), stdin=subprocess.PIPE, stdout=subprocess.PIPE, startupinfo=si, creationflags=subprocess.CREATE_NEW_CONSOLE):
pass
This way child.py can print to stderr even if run from a GUI script.
There should be a way to selectively set STARTUPINFO.hStdInput/hStdOutput/hStdError to None using Popen arguments.
CPython versions tested on:
3.10
Operating systems tested on:
Windows
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at Lib/subprocess.py around the linked line 1398 and reproduce the parent.py/child.py case on Windows using pythonw.exe and CREATE_NEW_CONSOLE. Done means Popen can pipe stdin and stdout while selectively leaving stderr as the default handle, including when the parent is a GUI script.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100