python / python/cpython

asyncio: BaseSelectorEventLoop._write_to_self swallows OSError, so a loop woken from another thread hangs silently when the self-pipe write is denied

Aberta
#156,054 1 comentário 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

stdlib topic-asyncio type-bug
Linguagem predominante
Python
Estrelas
77.2k
Forks
36k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

What happens

BaseSelectorEventLoop._write_to_self() writes one byte to the loop's self-pipe to wake the
selector. It catches and discards the write error by design:

def _write_to_self(self):
    csock = self._csock
    if csock is None:
        return
    try:
        csock.send(b'\0')
    except OSError:
        if self._debug:
            logger.debug("Fail to write a null byte into the self-pipe socket", exc_info=True)

If that send() cannot succeed, the wakeup is lost silently. Outside -X dev/debug mode there
is no traceback, no log line, and no non-zero exit.

Where that becomes a hang is any path whose progress depends on the wakeup arriving — in our case
executor completion, i.e. asyncio.to_thread() / run_in_executor(). call_soon_threadsafe() is
the API that calls _write_to_self(), but it is not a reliable symptom on its own: it queues the
callback before waking, so it can still be picked up (see the reproduction note below). The process
simply stops making progress, with nothing distinguishing it from a deadlock in user code.

We hit this in a seccomp-confined sandbox that denies send() (and sendall()) on an AF_UNIX
socketpair while permitting write()/os.write() on the very same file descriptor. The
practical effect was that our test suite stopped producing output at all — no failure, no partial
result, just a process killed later by an outer timeout.

Why it is worth reporting rather than working around

The silent-swallow is deliberate and defensible for a transient EAGAIN on a full pipe. It is much
less defensible for a persistent error such as EPERM/EACCES, where the loop is now permanently
unwakeable and nothing says so. The failure mode is indistinguishable from a deadlock in user code,
which is where we spent our debugging time.

Reproduction

Any environment where send() on the self-pipe socket is denied but write() is permitted. Minimal
shape:

import asyncio

async def main():
    return await asyncio.to_thread(lambda: "COMPLETED")

print(asyncio.run(main()))     # prints COMPLETED normally; hangs forever when send() is denied

Use asyncio.to_thread (or run_in_executor). Do NOT reduce this to a bare
loop.call_soon_threadsafe probe — it may PASS even with the write denied, and a pass there is not a
refutation of this report.
call_soon_threadsafe appends the callback to loop._ready before it
calls _write_to_self(), so a loop that is about to inspect _ready anyway can pick the callback up
without ever needing the wakeup. It wins a race that the executor-completion path loses. We measured
that smaller probe exiting 0 under the same denial that hangs the snippet above.

Measured, with a negative control in the same script and the patch as the only variable:

run result
unpatched, send() denied hung, killed at 25 s
_write_to_self monkeypatched to use os.write COMPLETED, exit 0

Reproduced twice by separate operators, on the same host and the same Python build, with the
sandbox as the only variable. We have NOT reproduced it on a second machine or a second Python
build
— so if you cannot reproduce it, the environment is the first thing to compare, not the
finding. To check whether yours is the same class of environment:

$ grep -E '^Seccomp' /proc/self/status     # 2 = SECCOMP_MODE_FILTER

Measured in our case: Seccomp: 2, Seccomp_filters: 1 inside the sandbox against Seccomp: 0
on the same host outside it.

⚠️ If you reproduce this in a container/sandbox, carry both a must-FAIL baseline and a must-PASS
control.
A misconfigured sandbox denies everything, which yields errors that look exactly like a
confirmation of this report. Without the must-pass control you cannot distinguish "the write was
denied" from "nothing ran at all" — that mistake cost us four probes and one failed cross-check.

Suggested direction (not a validated fix)

os.write(csock.fileno(), b'\0') succeeds where csock.send(b'\0') is denied, on the same fd, in
our environment. A plausible shape is to try the socket send and fall back, or to widen what the
handler treats as fatal so a persistent error surfaces instead of being discarded.

Honest scope — this is a reporter's suggested patch with a reproduction, not a validated fix:

  • Tested only as a monkeypatch in a probe process, on Linux, against BaseSelectorEventLoop
    only
    .
  • CPython's own test suite has not been run against it.
  • _csock is selector-loop specific; the proactor loop wakes itself differently, so this needs a
    fallback rather than a straight substitution. (Two of us reached that conclusion independently.)
  • We have not surveyed which other platforms or socket types would be affected.

If the maintainers would prefer the error surfaced rather than the write changed, that seems equally
reasonable to us — the part we care about is that a permanently unwakeable loop should not be silent.

Guia de contribuição

Abrir o guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Direção de pesquisa

Comece em asyncio's BaseSelectorEventLoop._write_to_self() e rastreie o caminho de conclusão do executor usado por asyncio.to_thread() ou run_in_executor(). Reproduza com a negação de seccomp fornecida e os controles must-fail/must-pass; em seguida, execute os testes CPython asyncio verificando o comportamento de selector e proactor. Considera-se concluído quando uma falha persistente de wakeup não causar mais um travamento silencioso sem regredir o tratamento de erros transitórios.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python
Domínio
networking, operating-systems
Tipo de issue
Bug
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Ativa
Clareza
Razoavelmente clara
Facilidade para iniciantes
52/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.