MagicStack / MagicStack/uvloop
why run_coroutine_threadsafe so slow?
Nobody has claimed this yet.
- Dominant language
- Cython
- Stars
- 11.9k
- Forks
- 616
- PR merge metrics
- No merged PRs in 30d
Description
- uvloop version: 0.14.0
- Python version: 3.6.5
- Platform: centos 6.3 cpu:Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
I have a need to run my code in two loop, demo code like below
import threading
import asyncio
import time
import traceback
import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
g_loop = asyncio.get_event_loop()
print("g_loop", id(g_loop), g_loop)
async def my_sum():
result = 0
for i in range(1000):
result += 1
async def func():
global g_loop
try:
while True:
s = time.time()
# await my_sum()
future = asyncio.run_coroutine_threadsafe(my_sum(), g_loop)
result = future.result(1)
e = time.time()
print("total", (e - s) * 1000 * 1000)
await asyncio.sleep(0.00001)
except Exception as e:
print("in this======", traceback.format_exc())
def thread_loop():
loop = asyncio.new_event_loop()
print("local loop", id(loop))
task = loop.create_task(func())
loop.run_forever()
t = threading.Thread(target=thread_loop)
t.start()
g_loop.run_forever()
t.join()
then, the print time cost about 100us ~ 150us
total 107.52677917480469
total 100.13580322265625
total 154.97207641601562
total 144.48165893554688
total 141.62063598632812
total 135.18333435058594
total 174.99923706054688
total 99.89738464355469
total 174.7608184814453
total 102.04315185546875
but when I run my code in one loop
import threading
import asyncio
import time
import traceback
import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
g_loop = asyncio.get_event_loop()
print("g_loop", id(g_loop), g_loop)
async def my_sum():
result = 0
for i in range(1000):
result += 1
async def func():
global g_loop
try:
while True:
s = time.time()
await my_sum()
# future = asyncio.run_coroutine_threadsafe(my_sum(), g_loop)
# result = future.result(1)
e = time.time()
print("total", (e - s) * 1000 * 1000)
await asyncio.sleep(0.00001)
except Exception as e:
print("in this======", traceback.format_exc())
def thread_loop():
loop = asyncio.new_event_loop()
print("local loop", id(loop))
task = loop.create_task(func())
loop.run_forever()
t = threading.Thread(target=thread_loop)
t.start()
g_loop.run_forever()
t.join()
the time cost about 40us ~ 80us
total 40.0543212890625
total 39.577484130859375
total 39.577484130859375
total 39.577484130859375
total 76.2939453125
total 40.0543212890625
total 39.10064697265625
total 81.53915405273438
total 40.0543212890625
Does change a thread need extra 60us ~ 70us? why so slow?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied two-loop and one-loop examples with uvloop 0.14.0, Python 3.6.5, and the stated platform, then compare the timing around asyncio.run_coroutine_threadsafe and await my_sum(). Done means identifying the source of the additional cross-thread latency or documenting why it is expected, with a focused reproduction or benchmark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100