python / python/cpython

asyncio subprocess spawn does not record the awaiting task

Open
#157,259 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib topic-asyncio type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

The internal subprocess pipe-connect task registers no awaited_by edge

Repro:

import asyncio, sys

async def main():
    spawn = asyncio.create_task(
        asyncio.create_subprocess_exec(sys.executable, '-c', 'pass',
                                       stdin=asyncio.subprocess.PIPE),
        name='spawn')

    await asyncio.sleep(0)
    await asyncio.sleep(0)

    for t in asyncio.all_tasks() - {asyncio.current_task()}:
        asyncio.print_call_graph(t)

    proc = await spawn
    await proc.wait()

asyncio.run(main())

Actual:

* Task(name='spawn', id=0x103e00b90)
  + Call stack:
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/unix_events.py', line 211, in async _UnixSelectorEventLoop._make_subprocess_transport()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py', line 1839, in async BaseEventLoop.subprocess_exec()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/subprocess.py', line 249, in async create_subprocess_exec()
* Task(name='Task-2', id=0x103b7d0a0)
  + Call stack:
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py', line 1753, in async BaseEventLoop.connect_write_pipe()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_subprocess.py', line 188, in async BaseSubprocessTransport._connect_pipes()

Expected:

* Task(name='spawn', id=0x105620b90)
  + Call stack:
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/unix_events.py', line 211, in async _UnixSelectorEventLoop._make_subprocess_transport()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py', line 1839, in async BaseEventLoop.subprocess_exec()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/subprocess.py', line 249, in async create_subprocess_exec()
* Task(name='Task-2', id=0x10539d0a0)
  + Call stack:
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py', line 1753, in async BaseEventLoop.connect_write_pipe()
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_subprocess.py', line 194, in async BaseSubprocessTransport._connect_pipes()
  + Awaited by:
    * Task(name='spawn', id=0x105620b90)
      + Call stack:
      |   File '/Users/timofeiivankov/cpython/Lib/asyncio/unix_events.py', line 211, in async _UnixSelectorEventLoop._make_subprocess_transport()
      |   File '/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py', line 1839, in async BaseEventLoop.subprocess_exec()
      |   File '/Users/timofeiivankov/cpython/Lib/asyncio/subprocess.py', line 249, in async create_subprocess_exec()

Proposed fix: register the edge in BaseSubprocessTransport.__init__ and discard it in doe callback

I have a fix ready for that

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Lib/asyncio/base_subprocess.py at BaseSubprocessTransport.init and inspect the subprocess pipe-connection path shown in the traceback. Run the supplied asyncio reproduction and confirm that the internal pipe-connect task reports an “Awaited by” edge for the spawn task, then verify the edge is discarded by the completion callback.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.