cloudflare / cloudflare/workerd

🐛 Bug Report — Runtime APIs: Socket `closed`/stream-EOF promises never settle at IoContext teardown, permanently leaking AsyncLocalStorage frames via pending reactions

Open
#7,202 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.7k
Forks
739
Avg merge
2d 20h
Merged PRs (30d)
174

Description

## Summary

When an invocation ends, workerd tears down its `cloudflare:sockets` sockets without settling `socket.closed` or the readable stream's EOF promise. Any reaction registered on those promises stays pending forever. Because promise reactions capture the active `AsyncContextFrame`, a reaction registered inside an `AsyncLocalStorage.run(...)` scope permanently retains the whole stored context graph.

With a database driver that subscribes to these promises per connection (postgres.js's Cloudflare build does: `closed.then(...)`, reader acquisition, a read loop with an always-pending `reader.read()`), every invocation leaks its full context — in our production-shaped worker, ~145 KB per scheduled invocation, monotonic through GC, until the 128 MB isolate limit.

## Environment

- workerd via wrangler 4.84.1, local dev (reproduced)
- Production Workers (observed, identical slope)
- `nodejs_compat`
- Compatibility date `2026-04-20`
- Sockets opened through a Hyperdrive binding's connection string, using postgres.js 3.4.9

## Retainer chain

From V8 heap snapshots, with a forced GC between samples; every hop verified:

```
(Global handles)
-> pending PromiseReaction
-> socket `closed` promise (held by its workerd Resolver / the stream's eofResolverPair)
-> captured AsyncContextFrame
-> StorageEntry
-> the AsyncLocalStorage value (our per-request context object)
-> everything it references
```

## Reproduction sketch

In a worker with `nodejs_compat`:

```js
import { AsyncLocalStorage } from 'node:async_hooks';
import { connect } from 'cloudflare:sockets';

const als = new AsyncLocalStorage();

export default {
async scheduled() {
await als.run({ big: new Uint8Array(100_000) }, async () => {
const socket = connect({ hostname: 'example.com', port: 5432 });
socket.closed.then(() => {});
// invocation ends without closing the socket explicitly
});
},
};
```

Let the invocation end without closing the socket explicitly (or even after `close()` in some teardown orders). Drive N invocations, force GC, take a snapshot: one pinned `AsyncContextFrame` per invocation, held by the pending reaction.

Registering the same handler through an `AsyncLocalStorage.snapshot()` captured at module scope (an empty frame) eliminates the growth — that is the workaround we ship today, as a patch to the driver.

## Expected

Sockets destroyed at IoContext teardown settle their `closed` and EOF promises (rejecting with a teardown error, matching the behavior documented for I/O outliving a request), so reactions run and release their frames.

## Impact

Any long-lived isolate — cron-driven Workers especially — using `AsyncLocalStorage` together with a socket-based driver leaks until OOM.

Possibly related: #5665 (unresolved promises retained in streaming paths) may be a sibling of the same teardown-settlement gap, though the growth here is monotonic across forced GCs and across invocations, rather than a matter of promise volume within a single request.

Contributor guide

Open the contributing guide

Research direction

Start by tracing cloudflare:sockets connection teardown through IoContext teardown, focusing on the socket.closed promise and the readable stream EOF promise. Reproduce the AsyncLocalStorage retention sketch, then verify that teardown settles both promises and that forced-GC heap snapshots no longer show one retained AsyncContextFrame per invocation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, javascript
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
47/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.