Asyncio: fork() in coroutine causes bad file descriptor

Bug report

Bug description:

Here is a simple repro:

import asyncio
import os

async def main():
    pid = os.fork()
    if pid:
        os.waitpid(pid, 0)

asyncio.run(main())

The traceback looks like:

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 641, in run_until_complete
    self.run_forever()
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
    self._run_once()
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1898, in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 566, in select
    kev_list = self._selector.control(None, max_ev, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/forkasyncio.py", line 9, in <module>
    asyncio.run(main())
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 189, in run
    with Runner(debug=debug) as runner:
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 63, in __exit__
    self.close()
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 72, in close
    loop.run_until_complete(loop.shutdown_asyncgens())
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 641, in run_until_complete
    self.run_forever()
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
    self._run_once()
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1898, in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 566, in select
    kev_list = self._selector.control(None, max_ev, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 9] Bad file descriptor
sys:1: RuntimeWarning: coroutine 'BaseEventLoop.shutdown_asyncgens' was never awaited

I believe the traceback is produced by the child process.

I would not expect bad file descriptor in either the parent or the child because fork is supposed to copy the open file descriptors. I have even tried a custom fork module to ensure os.fork isn't closing the file descriptor.

The issue also happens if the child does sys.exit(). I find the best workaround is to have the child do os._exit(0):

import asyncio
import os
import sys

async def main():
    pid = os.fork()
    if pid:
        os.waitpid(pid, 0)
    else:
        sys.stdout.flush()
        sys.stderr.flush()
        os._exit(0)

asyncio.run(main())

This produces no traceback. I also avoid doing anything asyncio in the child.

I'm mainly curious about what file descriptor is bad; it doesn't seem possible.

It sounds like this issue could be fixed by https://github.com/python/cpython/pull/99539, but I still reproduce the issue in 3.12.7 and 3.13.0, though with a slightly different exception:

Traceback (most recent call last):
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 1948, in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py", line 566, in select
    kev_list = self._selector.control(None, max_ev, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: I/O operation on closed kqueue object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 71, in close
    loop.run_until_complete(loop.shutdown_asyncgens())
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py", line 1948, in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py", line 566, in select
    kev_list = self._selector.control(None, max_ev, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: I/O operation on closed kqueue object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/forkasyncio.py", line 10, in <module>
    asyncio.run(main())
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 193, in run
    with Runner(debug=debug, loop_factory=loop_factory) as runner:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 62, in __exit__
    self.close()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/runners.py", line 77, in close
    loop.close()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/unix_events.py", line 68, in close
    super().close()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py", line 104, in close
    self._close_self_pipe()
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py", line 111, in _close_self_pipe
    self._remove_reader(self._ssock.fileno())
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/selector_events.py", line 305, in _remove_reader
    self._selector.unregister(fd)
  File "/Users/will/.pyenv/versions/3.12.7/lib/python3.12/selectors.py", line 542, in unregister
    self._selector.control([kev], 0, 0)
ValueError: I/O operation on closed kqueue object
/Users/will/.pyenv/versions/3.12.7/lib/python3.12/asyncio/base_events.py:712: RuntimeWarning: coroutine 'BaseEventLoop.shutdown_asyncgens' was never awaited

CPython versions tested on:

3.11

Operating systems tested on:

macOS

Start with the supplied macOS reproduction using asyncio.run(main()) and os.fork(). Trace the failure through Lib/asyncio/runners.py, base_events.py, unix_events.py, selector_events.py, and selectors.py, comparing the 3.11 and 3.12/3.13 tracebacks. Done should identify the child-process file-descriptor or selector state involved and include regression coverage for the reported behavior.