MagicStack / MagicStack/uvloop
`asyncio.create_subprocess_exec`: cannot read `/dev/stdin`
Nobody has claimed this yet.
- Dominant language
- Cython
- Stars
- 11.9k
- Forks
- 616
- PR merge metrics
- No merged PRs in 30d
Description
* **uvloop version**: `uvloop==0.17.0`
* **Python version**: `Python 3.10.10`
* **Platform**: `Linux 0af9a1603f81 5.10.76-linuxkit #1 SMP Mon Nov 8 10:21:19 UTC 2021 x86_64 GNU/Linux` (in a Docker container)
* **Can you reproduce the bug with `PYTHONASYNCIODEBUG` in env?**: yes
* **Does uvloop behave differently from vanilla asyncio? How?**: yes. Vanilla `asyncio` had a similar issue patched: https://github.com/python/cpython/issues/90522
Processes run with `asyncio.create_subprocess_exec` fail to read from `/dev/stdin` with `uvloop` but not with `asyncio`
`asyncio` had a similar issue fixed recently (in January 2022): https://github.com/python/cpython/issues/90522
This was backported to 3.10 and 3.11
I can reproduce this on Linux in a Docker container, with either Mac OS or Linux as hosts
I cannot reproduce this on Mac OS (seems to work fine)
Logs:
```
# PYTHONASYNCIODEBUG=1 python test.py
cat: /dev/stdin: No such device or address
Traceback (most recent call last):
File "//test.py", line 16, in
asyncio.run(f())
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "//test.py", line 13, in f
stdout, _ = await proc.communicate("test".encode("utf-8"))
File "/usr/local/lib/python3.10/asyncio/subprocess.py", line 195, in communicate
stdin, stdout, stderr = await tasks.gather(stdin, stdout, stderr)
File "/usr/local/lib/python3.10/asyncio/subprocess.py", line 147, in _feed_stdin
self.stdin.write(input)
File "/usr/local/lib/python3.10/asyncio/streams.py", line 325, in write
self._transport.write(data)
File "uvloop/handles/stream.pyx", line 674, in uvloop.loop.UVStream.write
File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive
RuntimeError: unable to perform operation on ; the handler is closed
```
Expected (using `asyncio`):
```
# PYTHONASYNCIODEBUG=1 python test.py
b'test'
```
Code to reproduce:
```py
import asyncio
# Commenting out `uvloop` below fixes the issue
import uvloop
uvloop.install()
async def f():
proc = await asyncio.create_subprocess_exec(
"cat",
"/dev/stdin",
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
)
stdout, _ = await proc.communicate("test".encode("utf-8"))
print(stdout)
asyncio.run(f())
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproducer in test.py and compare its behavior with vanilla asyncio on Linux in Docker. Trace subprocess stdin handling through uvloop/loop.pyx, uvloop/handles/stream.pyx, and uvloop/handles/handle.pyx. Done means the `/dev/stdin` subprocess prints `b'test'` under uvloop without the closed-transport error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100