python / python/cpython

asyncio: SelectorEventLoop busy-loops at 100% CPU forever when the self-pipe socketpair reaches EOF

オープン
#156,344 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

stdlib topic-asyncio type-bug
主要言語
Python
スター
77.2k
フォーク
35.9k
PR マージ指標
PR 指標を取得中

説明

Bug description

The selector-side twin of #156333. BaseSelectorEventLoop also wakes itself through a self-socket created by socket.socketpair(). On Windows this is a loopback TCP pair, and if the connection reaches a clean EOF while the loop runs, _read_from_self returns but the reader is still registered on the dead fd — select() reports it readable immediately, forever, pinning one core at 100% with nothing logged.

Lib/asyncio/selector_events.py:

def _read_from_self(self):
    while True:
        try:
            data = self._ssock.recv(4096)
            if not data:
                break
            self._process_self_data(data)
        except InterruptedError:
            continue
        except BlockingIOError:
            break

At EOF, recv() returns b'', the loop breaks, and the callback returns. But the fd is still registered via _add_reader(self._ssock.fileno(), self._read_from_self) from _make_self_pipe(). A closed-for-read socket is permanently "readable", so every select() iteration re-fires the callback — a tight spin. The loop is otherwise idle and never recovers.

The proactor half of this bug is fixed by #156343 (rebuild the pair on the empty result). The same OS teardown of the loopback pair that triggers it triggers this on any WindowsSelectorEventLoopPolicy process.

Reproducer (deterministic, Windows)
import asyncio, socket, time

calls = 0
orig = asyncio.selector_events.BaseSelectorEventLoop._read_from_self
def counting(self):
    global calls
    calls += 1
    return orig(self)
asyncio.selector_events.BaseSelectorEventLoop._read_from_self = counting

async def main():
    loop = asyncio.get_running_loop()
    await asyncio.sleep(0.1)
    loop._csock.shutdown(socket.SHUT_WR)   # graceful half-close = clean EOF
    global calls
    calls = 0
    start = time.process_time()
    await asyncio.sleep(3)
    print(f"_read_from_self calls during 3s idle: {calls}")
    print(f"CPU consumed while sleeping 3s: {time.process_time() - start:.2f}s")

loop = asyncio.SelectorEventLoop()
asyncio.set_event_loop(loop)
loop.run_until_complete(main())

Measured on Windows 11, CPython main (3.16.0a0, self-built): 582,692 callback invocations, 1.23s CPU, during a 3-second idle sleep. Expected ~0.

Note the instrumentation must patch the class before the loop is created — _make_self_pipe() captures the bound method at registration time, so patching an instance afterwards does not intercept the already-armed reader (which is also why the spin is invisible to profilers that hook late).

Environment
  • Windows 11, self-built main (3.16.0a0)
  • The proactor variant (#156333) reproduces on installed 3.12.10 / 3.13.12; the _read_from_self code path shown above is unchanged on main
Notes on scope
  • Unix loops use an AF_UNIX pair, which the OS does not tear down this way, so the organic trigger is Windows-specific; a deterministic test using shutdown(SHUT_WR) works on any platform, though.
  • Related, on the proactor side: the same teardown can also surface as ConnectionResetError from the pending recv instead of a clean EOF; that path is currently unhandled too (noted in #156343).
Suggested direction

On recv() == b'', rebuild the self-pipe the way #156343 does for the proactor loop: allocate the new pair first, move any signal wakeup fd registration (set_wakeup_fd() returns the previous fd, so it can be probed and moved only when it names the old socket — Unix loops with signal handlers), remove the old reader, close the old sockets, and register the reader on the new fd.

Linked PRs
  • gh-156345

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

Lib/asyncio/selector_events.py の _read_from_self と _make_self_pipe から始め、#156343 の proactor 修正と、#156345 に関連する作業を比較してください。Windows の self-pipe EOF ケースを再現し、アイドル状態の selector loop が reader を繰り返し呼び出さなくなっていることを、signal wakeup の登録が正しいままであることと併せて確認してください。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
backend, operating-systems
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
明確に書かれている
初心者へのやさしさ
25/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。