Some points may need reconsideration
- Dominant language
- Python
- Stars
- 302
- Forks
- 43
- PR merge metrics
- No merged PRs in 30d
Description
I’m reading this tutorial right now, particularly [the second part](https://www.roguelynn.com/words/asyncio-true-concurrency/). I like it overall, but I believe some things might be not entirely correct. Even though this tutorial is 4 years old, it's still valuable as there are not many advanced `asyncio` tutorials around the web. I believe it deserves some discussion.
I'm concerned with that part, where the author says that [one of the first examples](https://github.com/econchick/mayhem/blob/master/part-0/mayhem_4.py) of the Mayhem service is blocking.
>“We **are** essentially blocking ourselves; first we produce all the messages, one by one. Then we consume them, one by one.”
The author also presents [a synchronous example](https://github.com/econchick/mayhem/blob/master/part-1/mayhem_1.py) and says it’s equivalent to the async one (in the sense that they’re both non-concurrent). Then [she suggests](https://github.com/econchick/mayhem/blob/master/part-1/mayhem_2.py) wrapping every `await queue.put(msg)` in a separate task to make it really concurrent.
1. Wrapping every `queue.put(msg)` with a task is not what makes this example actually concurrent. In fact, it's a short sleep inside the `for` loop. If you only add a `sleep` call, the producer and the consumer will start working concurrently (because they are two separate coroutines created with `create_task` in the first place). As far as I understand from looking at the source code, `await queue.put(msg)` does not yield control to the event loop, unless the queue is full. If the queue has free slots, this method delegates to `put_nowait`, which is a regular sync method. This gives no time to the consumer coroutine to do any work. In the meantime, if you add a `sleep` to the sync example, nothing will change, it will indeed produce all the messages and only then consume them.
2. The author says that at some point the queue may be full, and awaiting `put` call will block the producer. That’s true, but I believe this is fine. I might be mistaken, but isn’t this an example of [backpressure](https://lucumr.pocoo.org/2020/1/1/async-pressure/)? If your consumer(s) cannot process the flood of events in a timely fashion, then you **want** to stop producing. Otherwise, you may end up creating thousands of tasks, all awaiting the queue, and at some point, your producer will run out of memory.
This is just my two cents, I would be happy to hear other opinions (maybe @econchick will jump in?). I'll keep on reading anyway, and maybe add some notes later.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.