micropython / micropython/micropython

asyncio: MicroPython vs. CPython create_task() garbage collection behavior

Open
#12,299 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

docs
Dominant language
C
Stars
22.1k
Forks
9k
Avg merge
6d 4h
Merged PRs (30d)
16

Description

(Not sure if I should label this a docs bug, feature request, or discussion. Apologies if I missed the mark.)

CPython exhibits a bit of a counterintuitive behavior, which requires users to store the result of asyncio.create_task(), otherwise face the danger of the task being garbage collected.

Will McGugan of Textual blogged about this here:
https://textual.textualize.io/blog/2023/02/11/the-heisenbug-lurking-in-your-async-code/

Vincent Bernat reported this as a documentation bug against CPython, https://github.com/python/cpython/issues/88831 last year, and subsequently the CPython documentation was adjusted to say:

Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done. For reliable “fire-and-forget” background tasks, gather them in a collection:

background_tasks = set()

for i in range(10):
    task = asyncio.create_task(some_coro(param=i))

    # Add task to the set. This creates a strong reference.
    background_tasks.add(task)

    # To prevent keeping references to finished tasks forever,
    # make each task remove its own reference from the set after
    # completion:
    task.add_done_callback(background_tasks.discard)

Is MicroPython exhibiting the same behavior? If so, should the documentation be adjusted?

  1. The MicroPython documentation provides an example that does not store a reference, basically what CPython warns users not to do. Is this a misleading example or a non-issue in MicroPython? Should the example be CPython-friendly anyway though? Should the documentation explain this difference?
  2. Even if one wanted to be better-safe-than-sorry and write portable code, the CPython-recommended code above does not work under MicroPython, resulting in a TypeError: unsupported type for __hash__: 'Task'. Should __hash__ be added for Tasks? Or should the documentation mention some other way to do this?

Not a MicroPython bug per se, but also worth mentioning: @peterhinch's (excellent!) async tutorial also has examples where Tasks are not stored. At one point it's mentioned that "[t]he .create_task method returns the Task instance which may be saved for status checking or cancellation" (emphasis mine). May, or should?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with MicroPython's asyncio documentation and its create_task example, then compare the behavior with CPython's asyncio documentation and issue 88831. Check the linked async tutorial and determine whether task references, portability guidance, or Task hashing need clarification; done means the behavioral difference and recommended documentation change are clearly established.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, embedded-iot
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.