cloudflare / cloudflare/workerd

Single TCP socket write() larger than 64 KiB silently destroys the connection (production runtime only)

Open
#7,074 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.7k
Forks
739
Avg merge
2d 20h
Merged PRs (30d)
174

Description

## Summary

A single `writer.write()` on a `cloudflare:sockets` TCP socket deterministically destroys the connection when the chunk is **larger than 65,536 bytes**. Exactly 65,536 bytes succeeds; 65,537 fails, every time. The `write()` promise **resolves normally** — the failure only surfaces on the next `read()` as `Error: Network connection lost.` (or, with some larger payloads, as a hang until "script will never generate a response"). Writing the identical bytes as multiple chunks of ≤ 64 KiB always succeeds.

- Reproduces: deployed workers and `wrangler dev --remote` (production runtime).
- Does **not** reproduce: local `wrangler dev`, any size (256 KiB single writes pass).
- Independent of `nodejs_compat` (repro doesn't use it), TLS (plaintext), destination (reproduced against two unrelated peers: `httpbin.org:80` and Supabase's Supavisor pooler `aws-1-eu-west-2.pooler.supabase.com:6543`), and compatibility_date (tested 2026-05-14 and 2026-08-01).
- Not documented as a limit on the TCP sockets page.

## Minimal repro (self-contained, no dependencies)

One plain-JS worker: it opens a TCP connection to `httpbin.org:80`, writes an ordinary HTTP POST (124-byte header + `size`-byte body) as **one** `writer.write()` (or split into `chunks` writes), then reads the response status line.

wrangler.toml + src/index.js

```toml
name = "socket-write-repro"
main = "src/index.js"
compatibility_date = "2026-08-01"
workers_dev = true
```

```js
import { connect } from 'cloudflare:sockets'

function timeout(ms, label) {
return new Promise((_, reject) => setTimeout(() => reject(new Error(`TIMEOUT ${label} after ${ms}ms`)), ms))
}

export default {
async fetch(req) {
const u = new URL(req.url)
const size = Number(u.searchParams.get('size') ?? 65536)
const chunks = Number(u.searchParams.get('chunks') ?? 1)
const host = u.searchParams.get('host') ?? 'httpbin.org'
const log = []

const body = new Uint8Array(size).fill(120)
const head = new TextEncoder().encode(
`POST /post HTTP/1.1\r\nHost: ${host}\r\nContent-Type: application/octet-stream\r\nContent-Length: ${size}\r\nConnection: close\r\n\r\n`,
)
const payload = new Uint8Array(head.length + size)
payload.set(head, 0)
payload.set(body, head.length)

log.push(`target=${host}:80 payloadBytes=${payload.length} chunks=${chunks}`)

let sock
try {
sock = connect(`${host}:80`)
const writer = sock.writable.getWriter()
const reader = sock.readable.getReader()

const t0 = Date.now()
if (chunks === 1) {
await Promise.race([writer.write(payload), timeout(10000, 'write')])
} else {
const per = Math.ceil(payload.length / chunks)
for (let i = 0; i < payload.length; i += per) {
await Promise.race([writer.write(payload.subarray(i, i + per)), timeout(10000, 'write')])
}
}
log.push(`WRITE OK in ${Date.now() - t0}ms`)

const dec = new TextDecoder()
let acc = ''
while (true) {
const r = await Promise.race([reader.read(), timeout(10000, 'read')])
if (r.done) {
log.push(`socket closed by peer, got ${acc.length} response bytes`)
break
}
acc += dec.decode(r.value, { stream: true })
if (/^HTTP\/1\.1 (\d+)/.test(acc)) {
log.push(`HTTP RESPONSE: ${acc.slice(0, acc.indexOf('\r\n'))}`)
break
}
}
} catch (err) {
log.push(`ERROR: ${err?.constructor?.name}: ${err?.message}`)
} finally {
try {
sock?.close()
} catch {}
}
return new Response(`${log.join('\n')}\n`, { headers: { 'content-type': 'text/plain' } })
},
}
```

```sh
npx wrangler dev --remote --port 8789 # production runtime; local dev does NOT repro

curl 'http://127.0.0.1:8789/?size=65411&chunks=1' # 65,535 B single write -> HTTP/1.1 200 OK
curl 'http://127.0.0.1:8789/?size=65412&chunks=1' # 65,536 B single write -> HTTP/1.1 200 OK
curl 'http://127.0.0.1:8789/?size=65413&chunks=1' # 65,537 B single write -> Network connection lost
curl 'http://127.0.0.1:8789/?size=200000&chunks=1' # 200 KB single write -> Network connection lost
curl 'http://127.0.0.1:8789/?size=200000&chunks=7' # same bytes, 7 writes -> HTTP/1.1 200 OK
```

Observed output for the failing case (2026-08-20):

```
target=httpbin.org:80 payloadBytes=65537 chunks=1
WRITE OK in 0ms <- the write() promise resolved
ERROR: Error: Network connection lost.
```

## Expected

Either the write succeeds (streams should segment arbitrarily large chunks), or `write()` rejects with a documented error. Resolving the write and then destroying the connection makes the failure look like a server-side reset — which is exactly how it presented in production for us.

## Local-runtime comparison

The same worker pointed at a local byte-counting TCP sink under plain `wrangler dev` accepts single writes of 262,144 bytes without issue, so local workerd does not have the limit — repro requires `--remote` or a deployed worker.

12-line sink server used for the local comparison

```js
import net from 'node:net'

const port = Number(process.argv[2] ?? 9977)
net
.createServer((sock) => {
let total = 0
sock.on('data', (buf) => {
total += buf.length
sock.write(`${total}\n`)
})
})
.listen(port, () => console.log(`sink listening on ${port}`))
```

## Real-world impact

Postgres wire-protocol drivers send each protocol message as a single `write()`. postgres.js's Cloudflare build (`cf/polyfills.js`) does exactly that, so any SQL statement whose wire size exceeds 64 KiB — e.g. a multi-row INSERT — fails from a Worker with `CONNECTION_CLOSED` (or hangs), while the identical statement succeeds from Node against the same server. We hit this in an ETL worker whose bulk INSERTs of document metadata (~600 rows, ~230 KB per statement) could never persist; the job retried for days before we isolated the boundary. Chunking writes to ≤ 32 KiB inside the driver's socket shim fully fixes it. Possibly related undiagnosed report: cloudflare/workers-sdk#6179.

## Environment

- wrangler 4.124.0, via `wrangler dev --remote` and deployed workers
- compatibility_date tested: 2026-05-14 and 2026-08-01; no compatibility_flags
- Account is on the Workers Free plan — I found no documented plan-dependent socket limit, and total throughput is clearly not capped (200 KB in 7 chunks succeeds), but please flag it if per-write size is somehow plan-gated

Contributor guide

Open the contributing guide

Research direction

Start with the self-contained repro in wrangler.toml and src/index.js, running it with wrangler dev --remote to confirm the 65,536-byte boundary and comparing it with local wrangler dev. Trace the cloudflare:sockets write path in workerd and compare the behavior with the chunked writes and the postgres.js cf/polyfills.js case. Done means large single writes either succeed or reject explicitly without destroying the connection.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, javascript
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.