Inconsistent `Popen.communicate()` behavior if stdin/stdout/stderr is closed `PIPE`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
The behavior with Popen.communicate() if stdin/stdout/stderr are PIPEs and they are closed before calling communicate() is rather strange.
If both stdout and stderr are PIPEs and one (or both) is closed, communicate() succeeds. If only one of them is a PIPE and it is closed, the behavior changes and there's a ValueError instead:
>>> from subprocess import Popen, PIPE
>>>
>>> p = Popen(['python', '-V'], stdout=PIPE, stderr=PIPE)
>>> p.stdout.close()
>>> p.communicate()
(b'', b'')
>>>
>>> p = Popen(['python', '-V'], stdout=PIPE)
>>> p.stdout.close()
>>> p.communicate()
Traceback (most recent call last):
File "<python-input-7>", line 1, in <module>
p.communicate()
~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/subprocess.py", line 1207, in communicate
stdout = self.stdout.read()
ValueError: read of closed file
If stdin is a closed PIPE, the behavior is pretty much the opposite of the above. If other streams are not PIPEs, communicate() succeeds, but if the are other PIPEs, we get a ValueError:
>>> from subprocess import Popen, PIPE
>>>
>>> p = Popen(['python', '-c', 'pass'], stdin=PIPE)
>>> p.stdin.close()
>>> p.communicate()
(None, None)
>>>
>>> p = Popen(['python', '-c', 'pass'], stdin=PIPE, stdout=PIPE)
>>> p.stdin.close()
>>> p.communicate()
Traceback (most recent call last):
File "<python-input-14>", line 1, in <module>
p.communicate()
~~~~~~~~~~~~~^^
File "/usr/lib/python3.14/subprocess.py", line 1220, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/subprocess.py", line 2081, in _communicate
self.stdin.flush()
~~~~~~~~~~~~~~~~^^
ValueError: flush of closed file
This inconsistency seems to be caused by an optimization that leads to two different code paths depending on the number of PIPEs. I needed to spend some time to understand why our tests behaved seemingly inconsistently after code was refactored.
It would be easy to fix problems by changing code like if self.stdout: to if self.stdout and not self.stdout.closed:. I guess it could be argued that raising a ValueError is the correct way to handle these situations, but in that case it should be raised always.
Tested with Python 3.14 alpha 5. Occurs also with earlier ones.
CPython versions tested on:
3.14
Operating systems tested on:
Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the closed-PIPE examples first, then inspect Lib/subprocess.py around the communicate() optimization linked near line 1194 and the stdin/stdout handling shown in the traceback. Decide and confirm the intended consistent behavior for closed streams, then add regression coverage for the affected combinations and verify the existing subprocess tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100