MagicStack / MagicStack/uvloop
Asynchronous multiprocessing.Pipe reads throw when using uvloop
Ninguém assumiu esta issue ainda.
- Linguagem predominante
- Cython
- Estrelas
- 11.9k
- Forks
- 616
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
Summary
When using an asyncio.Event() with loop.add_reader() to asynchronously read from a multiprocessing.Pipe, the call to recv_bytes() throws:
Process reader:
Traceback (most recent call last):
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/process.py", line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/james/code/connection_test.py", line 25, in reader_wrapper
uvloop.run(async_reader(read_pipe))
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/james/code/.venv/lib/python3.13/site-packages/uvloop/__init__.py", line 109, in run
return __asyncio.run(
~~~~~~~~~~~~~^
wrapper(),
^^^^^^^^^^
...<2 lines>...
**run_kwargs
^^^^^^^^^^^^
)
^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/home/james/code/.venv/lib/python3.13/site-packages/uvloop/__init__.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/home/james/code/connection_test.py", line 20, in async_reader
message = read_pipe.recv_bytes()
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 437, in _recv_bytes
return self._recv(size)
~~~~~~~~~~^^^^^^
File "/home/james/.pyenv/versions/3.13.5/lib/python3.13/multiprocessing/connection.py", line 395, in _recv
chunk = read(handle, remaining)
BlockingIOError: [Errno 11] Resource temporarily unavailable
This does not occur when running with asyncio
Environment
Python: 3.13.5
uvloop: 0.21.0
Reproduction
import asyncio
import uvloop
from multiprocessing import Process
from multiprocessing.connection import Connection, Pipe, wait
def writer(write_pipe: Connection):
message = b'!' * 65536
while True:
write_pipe.send_bytes(message)
async def async_reader(read_pipe: Connection):
message_available = asyncio.Event()
asyncio.get_running_loop().add_reader(
read_pipe.fileno(), message_available.set
)
while True:
await message_available.wait()
while read_pipe.poll():
message = read_pipe.recv_bytes()
print('Got message')
message_available.clear()
def reader_wrapper(read_pipe: Connection):
uvloop.run(async_reader(read_pipe))
if __name__ == '__main__':
read_pipe, write_pipe = Pipe(duplex=False)
reader_process = Process(
name="reader", target=reader_wrapper, args=(read_pipe,)
)
reader_process.start()
writer_process = Process(
name="writer", target=writer, args=(write_pipe,)
)
writer_process.start()
wait([reader_process.sentinel, writer_process.sentinel])
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Primeiros passos
- Leia a issue inteira e depois o guia de contribuição do projeto.
- Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
- Faça um fork do repositório e trabalhe em uma branch.
- Abra um pull request que referencie o número da issue.
Direção de pesquisa
Comece executando a reprodução fornecida com Python 3.13.5 e uvloop e compare seu comportamento com o loop de eventos do asyncio, concentrando-se em add_reader(), multiprocessing.Pipe, poll() e recv_bytes(). Considera-se concluído quando as leituras assíncronas de Pipe não gerarem mais BlockingIOError no uvloop, mantendo o comportamento de leitura demonstrado.
Escrita pelo modelo de indexação a partir do texto da issue.
Avaliação
- Stack de tecnologia
- python
- Domínio
- backend, networking
- Tipo de issue
- Bug
- Dificuldade
- 4/5
- Tempo estimado
- 3-5 dias
- Status de atividade
- Estagnada
- Clareza
- Razoavelmente clara
- Facilidade para iniciantes
- 35/100