reserve() permanently stranded when a pooled connection is terminated server-side while the reserve is queued (3.4.9)
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 8.7k
- Forks
- 374
- Avg merge
- 11d 16h
- Merged PRs (30d)
- 1
Description
Summary
A reserve() that is still queued (waiting for a pooled connection to free) is permanently stranded if the server terminates the connection it was waiting on. The promise returned by reserve() never settles — no resolve, no reject, no timeout — and the reserved slot is lost for the lifetime of the process.
Reproduced deterministically (3/3) on postgres@3.4.9 (current latest) against PostgreSQL 16.6, Node v22.18.0.
Mechanism
reserve() (src/index.js:203) pushes a { reserve, reject } pseudo-query into queries and waits. onopen() is the only place a reserve is resolved — src/index.js:410, if (query.reserve) return query.reserve(c).
When the connection the reserve is waiting on is closed by the server, onclose() runs:
// src/index.js:426
queries.length && connect(c, queries.shift())
This shifts the pending reserve out of queries and hands it to the dead connection's reconnect as initial. On the reconnect, initial being a reserve is dropped rather than resolved:
// src/connection.js:563 (fetch_types path, the default)
initial.reserve && (initial = null)
// src/connection.js:567 (non-fetch_types path)
initial && !initial.reserve && execute(initial) // reserve is explicitly NOT executed
So after reconnect the reserve is gone from queries (shifted out at index.js:426) and was never resolved (dropped at connection.js:563/567). onopen() fires on the healthy new connection, sees no matching queued reserve, and the original promise hangs forever.
Reproduction
Single file, no framework, no application code. max: 1 makes the queueing deterministic; any pool size reproduces it whenever a reserve is queued behind a connection that then dies.
import postgres from 'postgres'
const base = { host: '127.0.0.1', port: 5499, user: 'repro', database: 'postgres', max: 1 }
const withTimeout = (p, ms, label) =>
Promise.race([
p.then((v) => ({ ok: true, v })),
new Promise((res) => setTimeout(() => res({ ok: false, label, ms }), ms)),
])
const sql = postgres(base)
const admin = postgres({ ...base, max: 1 })
// 1. Take the single pooled connection.
const held = await sql.reserve()
const [{ pid }] = await held`select pg_backend_pid()::int as pid`
// 2. Queue a SECOND reserve() — it waits in the pool's `queries` array.
const pendingReserve = sql.reserve()
// 3. Server terminates the held backend (a failover, an idle reaper, or an admin).
await admin`select pg_terminate_backend(${pid})`
// 4. onclose() shifts the pending reserve out of `queries` and hands it to the
// dead connection's reconnect, which drops it. It never settles.
const outcome = await withTimeout(pendingReserve, 8000, 'pendingReserve')
console.log(outcome.ok
? 'pending reserve() RESOLVED — bug not reproduced'
: `pending reserve() NEVER SETTLED after ${outcome.ms} ms — STRANDED (bug reproduced)`)
await admin.end({ timeout: 1 }).catch(() => {})
process.exit(outcome.ok ? 1 : 0)
Output:
held reserved connection, backend pid=93939
second reserve() is now queued (pending)
server terminated backend pid=93939
RESULT: pending reserve() NEVER SETTLED after 8000 ms — STRANDED (bug reproduced)
Why it matters
reserve() is the documented way to pin a connection for a transaction or a session-scoped operation. A server-side termination of a pooled connection is routine in production — failover, an idle-connection reaper, idle_in_transaction_session_timeout, an admin pg_terminate_backend. When one lands while a reserve is queued, the caller's await sql.reserve() hangs with no error to catch and no timeout to trip, so a request handler waits forever and the reserved slot leaks. There is no application-side workaround short of racing every reserve() against a manual timeout.
Relationship to #751
This looks like the same root cause as #751, seen from the other side. #751 is the first-ever-connect path with fetch_types: false (connection.js returns before onopen); this is the reconnect-after-close path with fetch_types: true (the default). Both share one invariant: a reserve pseudo-query must remain in queries until onopen resolves it, and must never be consumed as a connection's initial. A fix that restores that invariant should close both.
Fix direction
I have a proven red repro but deliberately am not attaching a patch, because the naive one-line change at index.js:426 (reconnect empty instead of shifting the reserve) has to be coordinated with the reserve-drop at connection.js:563/567 and made safe against repeated close events, and that is your call to make in your own reconnect model. The shape that fixed it in my testing is: leave a head-of-queue reserve in queries on close and reconnect the socket empty, so onopen (index.js:410) resolves it the same way it resolves a fresh reserve — rather than routing it through initial, which drops it. Happy to open a PR along those lines if you'd like, and to fold in a regression test built on the repro above.
Versions: postgres 3.4.9, PostgreSQL 16.6, Node 22.18.0, macOS (aarch64). default_pipeline/prepared settings default; reproduced with fetch_types default (true).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the reserve queue and reconnect paths in src/index.js:203, 410, and 426, then compare the initial-reserve handling at src/connection.js:563 and 567. Use the supplied PostgreSQL termination reproduction as the failing case. Done means the queued reserve settles after reconnect instead of being dropped, with a regression test based on that reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs, postgresql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100