MagicStack / MagicStack/uvloop
Asynchronous multiprocessing.Pipe reads throw when using uvloop
未关闭
还没有人认领这个 Issue。
- 主要语言
- Cython
- 星标
- 11.9k
- 派生
- 616
- PR 合并指标
- 30 天内没有已合并 PR
描述
Summary
When using an asyncio.Event() with loop.add_reader() to asynchronously read from a multiprocessing.Pipe, the call to recv_bytes() throws:
Process reader:
Traceback (most recent call last):
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/james/code/connection_test.py", line 25, in reader_wrapper
uvloop.run(async_reader(read_pipe))
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/james/code/.venv/lib/python3.13/site-packages/uvloop/__init__.py", line 109, in run
return __asyncio.run(
~~~~~~~~~~~~~^
wrapper(),
^^^^^^^^^^
...<2 lines>...
**run_kwargs
^^^^^^^^^^^^
)
^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/home/james/code/.venv/lib/python3.13/site-packages/uvloop/__init__.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/home/james/code/connection_test.py", line 20, in async_reader
message = read_pipe.recv_bytes()
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 437, in _recv_bytes
return self._recv(size)
~~~~~~~~~~^^^^^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
chunk = read(handle, remaining)
BlockingIOError: [Errno 11] Resource temporarily unavailable
This does not occur when running with asyncio
Environment
Python: 3.13.5
uvloop: 0.21.0
Reproduction
import asyncio
import uvloop
from multiprocessing import Process
from multiprocessing.connection import Connection, Pipe, wait
def writer(write_pipe: Connection):
message = b'!' * 65536
while True:
write_pipe.send_bytes(message)
async def async_reader(read_pipe: Connection):
message_available = asyncio.Event()
asyncio.get_running_loop().add_reader(
read_pipe.fileno(), message_available.set
)
while True:
await message_available.wait()
while read_pipe.poll():
message = read_pipe.recv_bytes()
print('Got message')
message_available.clear()
def reader_wrapper(read_pipe: Connection):
uvloop.run(async_reader(read_pipe))
if __name__ == '__main__':
read_pipe, write_pipe = Pipe(duplex=False)
reader_process = Process(
name="reader", target=reader_wrapper, args=(read_pipe,)
)
reader_process.start()
writer_process = Process(
name="writer", target=writer, args=(write_pipe,)
)
writer_process.start()
wait([reader_process.sentinel, writer_process.sentinel])
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
首先,使用 uvloop 运行提供的 Python 3.13.5 复现代码,并将其行为与 asyncio 事件循环进行比较,重点关注 add_reader()、multiprocessing.Pipe、poll() 和 recv_bytes()。当 uvloop 下的异步 Pipe 读取不再引发 BlockingIOError,同时保留所演示的读取行为时,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- backend, networking
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100