MagicStack / MagicStack/uvloop
"RuntimeError: Racing with another loop to spawn a process" when spawning many processes from multiple threads
Personne n'a encore pris cette issue.
- Langage dominant
- Cython
- Étoiles
- 11.9k
- Forks
- 616
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
* **uvloop version**: `0.17.0`
* **Python version**: `3.10.8`
* **Platform**: Arch Linux
* **Can you reproduce the bug with `PYTHONASYNCIODEBUG` in env?**: Yes
* **Does uvloop behave differently from vanilla asyncio? How?**: Yes, there is no exception.
We have a multi-threaded application using `uvloop` that sometimes needs to spawn a lot of processes simultaneously.
This results in a lot of "RuntimeError: Racing with another loop to spawn a process" exceptions being raised.
The following is a minimal reproducible example that shows the problem:
```python
import asyncio
from threading import Thread
import uvloop
def create_processes(i):
async def inner():
processes = []
for _ in range(100):
p = await asyncio.create_subprocess_exec("true")
processes.append(p)
for p in processes:
await p.wait()
try:
asyncio.run(inner())
print(f"[{i}] Success.")
except Exception as e:
print(f"[{i}] Fail: {repr(e)}")
def main():
threads = []
for i in range(10):
t = Thread(target=lambda: create_processes(i))
t.start()
threads.append(t)
for t in threads:
t.join()
uvloop.install()
main()
```
When running this with `PYTHONASYNCIODEBUG=1` I get something like:
```
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.132 seconds
Executing took 0.133 seconds
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.142 seconds
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.147 seconds
[0] Fail: RuntimeError('Racing with another loop to spawn a process.')
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.157 seconds
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.175 seconds
Executing .inner() running at /home/tyilo/repos/uvloop-race/test.py:11> wait_for= cb=[run_until_complete..done_cb()] created at /usr/lib/python3.10/asyncio/tasks.py:636> took 0.176 seconds
[2] Fail: RuntimeError('Racing with another loop to spawn a process.')
[9] Fail: RuntimeError('Racing with another loop to spawn a process.')
[1] Fail: RuntimeError('Racing with another loop to spawn a process.')
[4] Fail: RuntimeError('Racing with another loop to spawn a process.')
[3] Success.
[6] Success.
[8] Success.
[5] Success.
[7] Success.
```
Without `uvloop` I get something like:
```
[3] Success.
[9] Success.
[5] Success.
[7] Success.
[2] Success.
[4] Success.
[8] Success.
[0] Success.
[6] Success.
[1] Success.
```
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Piste de recherche
Commencez par le reproducer minimal autour de asyncio.create_subprocess_exec, asyncio.run et uvloop.install, et comparez son comportement avec celui de vanilla asyncio sous PYTHONASYNCIODEBUG. Suivez le chemin de création des sous-processus utilisé par les boucles d’événements concurrentes ; c’est terminé lorsque le reproducer avec des threads peut créer tous les processus et attendre leur fin sans le RuntimeError dû à la course.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- backend, operating-systems
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 35/100