python / python/cpython

asyncio: BaseSelectorEventLoop._write_to_self swallows OSError, so a loop woken from another thread hangs silently when the self-pipe write is denied

未关闭
#156,054 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

stdlib topic-asyncio type-bug
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

What happens

BaseSelectorEventLoop._write_to_self() writes one byte to the loop's self-pipe to wake the
selector. It catches and discards the write error by design:

def _write_to_self(self):
    csock = self._csock
    if csock is None:
        return
    try:
        csock.send(b'\0')
    except OSError:
        if self._debug:
            logger.debug("Fail to write a null byte into the self-pipe socket", exc_info=True)

If that send() cannot succeed, the wakeup is lost silently. Outside -X dev/debug mode there
is no traceback, no log line, and no non-zero exit.

Where that becomes a hang is any path whose progress depends on the wakeup arriving — in our case
executor completion, i.e. asyncio.to_thread() / run_in_executor(). call_soon_threadsafe() is
the API that calls _write_to_self(), but it is not a reliable symptom on its own: it queues the
callback before waking, so it can still be picked up (see the reproduction note below). The process
simply stops making progress, with nothing distinguishing it from a deadlock in user code.

We hit this in a seccomp-confined sandbox that denies send() (and sendall()) on an AF_UNIX
socketpair while permitting write()/os.write() on the very same file descriptor. The
practical effect was that our test suite stopped producing output at all — no failure, no partial
result, just a process killed later by an outer timeout.

Why it is worth reporting rather than working around

The silent-swallow is deliberate and defensible for a transient EAGAIN on a full pipe. It is much
less defensible for a persistent error such as EPERM/EACCES, where the loop is now permanently
unwakeable and nothing says so. The failure mode is indistinguishable from a deadlock in user code,
which is where we spent our debugging time.

Reproduction

Any environment where send() on the self-pipe socket is denied but write() is permitted. Minimal
shape:

import asyncio

async def main():
    return await asyncio.to_thread(lambda: "COMPLETED")

print(asyncio.run(main()))     # prints COMPLETED normally; hangs forever when send() is denied

Use asyncio.to_thread (or run_in_executor). Do NOT reduce this to a bare
loop.call_soon_threadsafe probe — it may PASS even with the write denied, and a pass there is not a
refutation of this report.
call_soon_threadsafe appends the callback to loop._ready before it
calls _write_to_self(), so a loop that is about to inspect _ready anyway can pick the callback up
without ever needing the wakeup. It wins a race that the executor-completion path loses. We measured
that smaller probe exiting 0 under the same denial that hangs the snippet above.

Measured, with a negative control in the same script and the patch as the only variable:

run result
unpatched, send() denied hung, killed at 25 s
_write_to_self monkeypatched to use os.write COMPLETED, exit 0

Reproduced twice by separate operators, on the same host and the same Python build, with the
sandbox as the only variable. We have NOT reproduced it on a second machine or a second Python
build
— so if you cannot reproduce it, the environment is the first thing to compare, not the
finding. To check whether yours is the same class of environment:

$ grep -E '^Seccomp' /proc/self/status     # 2 = SECCOMP_MODE_FILTER

Measured in our case: Seccomp: 2, Seccomp_filters: 1 inside the sandbox against Seccomp: 0
on the same host outside it.

⚠️ If you reproduce this in a container/sandbox, carry both a must-FAIL baseline and a must-PASS
control.
A misconfigured sandbox denies everything, which yields errors that look exactly like a
confirmation of this report. Without the must-pass control you cannot distinguish "the write was
denied" from "nothing ran at all" — that mistake cost us four probes and one failed cross-check.

Suggested direction (not a validated fix)

os.write(csock.fileno(), b'\0') succeeds where csock.send(b'\0') is denied, on the same fd, in
our environment. A plausible shape is to try the socket send and fall back, or to widen what the
handler treats as fatal so a persistent error surfaces instead of being discarded.

Honest scope — this is a reporter's suggested patch with a reproduction, not a validated fix:

  • Tested only as a monkeypatch in a probe process, on Linux, against BaseSelectorEventLoop
    only
    .
  • CPython's own test suite has not been run against it.
  • _csock is selector-loop specific; the proactor loop wakes itself differently, so this needs a
    fallback rather than a straight substitution. (Two of us reached that conclusion independently.)
  • We have not surveyed which other platforms or socket types would be affected.

If the maintainers would prefer the error surfaced rather than the write changed, that seems equally
reasonable to us — the part we care about is that a permanently unwakeable loop should not be silent.

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 asyncio 的 BaseSelectorEventLoop._write_to_self() 开始,跟踪 asyncio.to_thread() 或 run_in_executor() 使用的 executor 完成路径。使用提供的 seccomp 拒绝和 must-fail/must-pass 控制进行复现,然后运行 CPython asyncio 测试,同时检查 selector 和 proactor 的行为。完成的标准是:持久性的 wakeup 失败不再导致静默挂起,同时不会使瞬时错误处理出现回归。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
networking, operating-systems
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
52/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。