porsager / porsager/postgres

nextWrite() throws an uncaughtException when a reserved connection's backend dies

Open Beginner friendly
#1,208 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
8.7k
Forks
374
Avg merge
11d 16h
Merged PRs (30d)
1

Description

Version

postgres 3.4.9 · Bun 1.4.0 · PostgreSQL 18.6 (also reproduces on 16)

What happens

A sql.reserve()d connection whose backend is terminated server-side kills the process on the next query:

TypeError: null is not an object (evaluating 'socket.write')
    at nextWrite (postgres/src/connection.js:255:15)

It surfaces as an uncaughtException, not a rejected promise — try/catch around the awaited query does not catch it, so there is no way for the application to defend against it.

Minimal reproduction
import postgres from 'postgres'

const sql = postgres(URL, { max: 2 })
const killer = postgres(URL, { max: 1 })

const reserved = await sql.reserve()
const [{ pid }] = await reserved`select pg_backend_pid() as pid`
await killer`select pg_terminate_backend(${pid})`

await reserved`select 1`.catch(() => {})   // rejects: CONNECTION_CLOSED  (correct)
await reserved`select 1`                   // <-- uncaughtException, process dies

Deterministic — first attempt, every time.

Why

closed() sets socket = null (connection.js#L448), and reconnect() only re-creates it on a later setTimeout (#L361) — so the connection sits with a null socket across ticks.

nextWrite() is the only consumer of socket in that file without a null guard. terminate() has if (socket) and end() has socket && …; this one dereferences directly.

It is also the one reached from the socket 'data' handler: the protocol dispatcher (L613–L847) and the SASL exchange (L688–L736) both call write(), and data is registered as socket.on('data', data) (L372). A throw there has no promise to reject, so it escapes to uncaughtException.

A pooled connection masks this, because the pool rotates a dead connection away. A reserved one is pinned and cannot be rotated, so every subsequent write on it is fatal. That is why this shows up as an unrecoverable crash loop specifically for reserve() users — in our case an advisory-lock helper, roughly one process exit every 8 minutes in production.

Worth noting how narrow the underlying race is without reserve(): ~2,800 synthetic connection kills (mid-handshake, blackholed handshakes, post-connect RSTs, >1024-byte statements to force the synchronous write branch) never reproduced it. Reserving is what makes it deterministic, because the pool can no longer heal it.

Suggested fix

Guard nextWrite() — and settle rather than drop. Returning false alone is not enough: the write vanishes and the caller awaits forever (verified — the query hangs).

function nextWrite(fn) {
  if (!socket) {
    chunk = null
    nextWriteTimer !== null && clearImmediate(nextWriteTimer)
    nextWriteTimer = null
    error(Errors.connection('CONNECTION_CLOSED', options, socket))
    return false
  }
  const x = socket.write(chunk, fn)
  ...

With this, the second query rejects with CONNECTION_CLOSED — catchable — and the process survives. We are running it as a patch against 3.4.9.

Happy to open a PR if the approach looks right.

Checked for duplicates

#896 is a different defect (query.origin undefined in queryError); #43 and #179 are pool/ECONNRESET reports without this stack.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/connection.js at nextWrite(), then read closed() and reconnect() to understand the null-socket window. Run the minimal reserved-connection reproduction and verify that a subsequent query rejects with CONNECTION_CLOSED, remains catchable, and does not escape as an uncaughtException.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, javascript, postgresql
Domain
backend, database
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.