agronholm / agronholm/anyio

UDPSocket.aclose() hangs on Windows when a send is in flight (regression from #1147)

Open
#1,237 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
260
Avg merge
1d 8h
Merged PRs (30d)
17

Description

### Summary

Since #1147 (4.14.0), `UDPSocket.aclose()` / `ConnectedUDPSocket.aclose()` await `self._protocol.closed_event`. On `ProactorEventLoop` there is a path where `connection_lost()` is never called, so that event is never set and `aclose()` waits forever.

It triggers whenever a datagram write is still in flight at close time — i.e. the ordinary "send a metric, then close" pattern.

### Repro

```python
import anyio

async def main() -> None:
sock = await anyio.create_connected_udp_socket("127.0.0.1", 9999)
await sock.send(b"x")
await sock.aclose() # never returns on Windows

anyio.run(main)
```

Nothing needs to be listening on the port.

Full reproducer with a CI matrix: https://github.com/graingert/anyio-udp-aclose-hang — the Windows jobs are red on purpose. Observed on `windows-latest`, CPython 3.12 and 3.13, anyio 4.14.2, with a `faulthandler` watchdog to turn the hang into a stack dump:

```
sent a datagram; calling aclose() ...
Timeout (0:00:30)!
File "...\asyncio\windows_events.py", line 774 in _poll
File "...\asyncio\windows_events.py", line 445 in select
File "...\asyncio\base_events.py", line 1961 in _run_once
...
File "...\anyio\_core\_eventloop.py", line 83 in run
File "...\repro_hang.py", line 26 in
```

The same scripts pass on ubuntu-latest, where `close()` schedules `_call_connection_lost` directly.

### Cause

`aclose()` waits for `connection_lost()`:

```python
async def aclose(self) -> None:
self._closed = True
if not self._transport.is_closing():
self._transport.close()

await self._protocol.closed_event.wait()
```

`_ProactorBasePipeTransport.close()` increments `_conn_lost`, and skips scheduling `_call_connection_lost` because a write is outstanding:

```python
def close(self):
if self._closing:
return
self._closing = True
self._conn_lost += 1
if not self._buffer and self._write_fut is None:
self._loop.call_soon(self._call_connection_lost, None)
```

Normally the write's done-callback would pick that up, but `_ProactorDatagramTransport._loop_writing()` returns on its first line, because `close()` has just incremented `_conn_lost`:

```python
def _loop_writing(self, fut=None):
try:
if self._conn_lost:
return # <-- bails out here

...
if not self._buffer or (self._conn_lost and self._address):
# The connection has been closed
if self._closing:
self._loop.call_soon(self._call_connection_lost, None) # <-- never reached
return
```

So `_call_connection_lost` is scheduled by neither path, `connection_lost()` is never called, `closed_event` stays unset, and the `await` never returns.

### Impact

Before 4.14 this path was a missed close — the `ResourceWarning` that #1147 set out to fix. From 4.14 the same path deadlocks, so shutdown code that closes a datagram socket after sending on it hangs instead. It surfaced in [anycorn](https://github.com/davidbrochart/anycorn), whose statsd logger sends a metric then closes its socket: after 4.11.0 → 4.14.2 its `windows-latest` jobs went from ~30s to hanging indefinitely, on 3.10 through 3.14.

### Workarounds

`aclose_forcefully()` cancels the wait, so the hang goes away — but `connection_lost()` still never runs, so the transport keeps its socket and the `ResourceWarning` returns (`repro_forcefully.py` reports `fileno = 556` after closing).

What does work is bypassing the wrapper on asyncio and driving the transport directly, so `abort()` is reachable (`repro_workaround.py`, green on Windows):

```python
transport.close()
await asyncio.sleep(0)
transport.abort()
with CancelScope(shield=True):
await protocol.closed.wait()
```

### Possible fix

`SocketStream.aclose()` already handles the equivalent case by following `close()` with `await sleep(0)` and then `abort()`. That works because `_force_close()`'s early-return guard tests `_called_connection_lost` rather than `_conn_lost`, so unlike `close()` it always schedules `_call_connection_lost`. The UDP wrappers have no equivalent step.

There may also be a CPython-side argument that `_loop_writing()`'s `_conn_lost` early return should not pre-empt the `_closing` handling below it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.