Unhandled rejection kills the process when the internal array-types query is cancelled (57014) - repro for #279
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 8.7k
- Forks
- 374
- Avg merge
- 11d 16h
- Merged PRs (30d)
- 1
Description
This is #279 - closed in 2022 for lack of a reproduction ("Let me know if you can repro this") - with a deterministic repro attached. The stack in that report already names the culprit frame (fetchArrayTypes -> ReadyForQuery).
Summary
When the server sends an ErrorResponse for the driver's internal array-types query - for example when that query is cancelled by statement_timeout or pg_cancel_backend (SQLSTATE 57014) - postgres.js emits an unhandledRejection that no application code can catch. Under Node's --unhandled-rejections=throw (the default on Vercel and in Node >= 15 for unhandled rejections) this kills the process, taking every in-flight request with it.
We hit this in production as repeated "whole site hangs" incidents: a statement gets cancelled, and the serverless instance dies with exit 128 instead of the one request failing.
Where it happens
src/connection.js, ReadyForQuery calls fetchArrayTypes() without awaiting it or attaching a handler:
if (needsTypes) {
initial.reserve && (initial = null)
return fetchArrayTypes() // connection.js:564 - promise dropped
}
async function fetchArrayTypes() { // connection.js:768
needsTypes = false
const types = await new Query([`
select b.oid, b.typarray
...
`], [], execute)
types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
}
If that Query rejects, the await throws and the plain promise returned by the async function rejects with no handler attached. Node reports exactly one unhandled rejection, top frame ErrorResponse (src/connection.js:815):
PostgresError: canceling statement due to statement timeout
at ErrorResponse (postgres/src/connection.js:815:30)
at handle (postgres/src/connection.js:489:6)
at Socket.data (postgres/src/connection.js:324:9)
Because the rejected promise belongs to no caller query, adding .catch() to every application query does not help (verified by promise-identity checks - it is an internal promise, not one of ours).
fetch_types: false avoids it entirely (verified with the repro below: 1 unhandled rejection with the default fetch_types: true, 0 with it disabled), since fetchArrayTypes is the only consumer of needsTypes - but that gives up array parsing.
The same shape exists in fetchState() (query.execute() at connection.js:810 is also dropped, reachable with target_session_attrs), and in the unawaited async auth handlers called from handle().
Reproduction
Deterministic, no pooler or special server needed. The only trick is making the array-types catalog query slower than a 1 ms statement_timeout, which the script does by creating throwaway composite types:
docker run -d --name pgrepro -e POSTGRES_PASSWORD=pg -e POSTGRES_DB=repro -p 55433:5432 postgres:16
node repro.mjs 'postgres://postgres:pg@127.0.0.1:55433/repro'
import postgres from 'postgres'
const DBURL = process.argv[2]
const SCHEMA = 'pgjs_dangling_repro'
const ARRAY_TYPES_SQL = `
select b.oid, b.typarray
from pg_catalog.pg_type a
left join pg_catalog.pg_type b on b.oid = a.typelem
where a.typcategory = 'A'
group by b.oid, b.typarray
order by b.oid`
const setup = postgres(DBURL, { max: 1, fetch_types: false, onnotice: () => {} })
const time = async () => {
const t = performance.now()
await setup.unsafe(ARRAY_TYPES_SQL)
return performance.now() - t
}
// make the driver's internal array-types query take > 1ms
await setup.unsafe(`CREATE SCHEMA IF NOT EXISTS ${SCHEMA}`)
let created = 0, elapsed = await time()
while (elapsed < 5 && created < 30000) {
let ddl = ''
for (let i = created; i < created + 2000; i++) ddl += `CREATE TYPE ${SCHEMA}.t${i} AS (a int);`
await setup.unsafe(ddl)
created += 2000
elapsed = await time()
}
console.log(`array-types query: ${elapsed.toFixed(1)}ms (${created} helper types)`)
const unhandled = []
process.on('unhandledRejection', (r) => unhandled.push(r))
// fetch_types defaults to true, so connection setup runs fetchArrayTypes();
// statement_timeout makes the server cancel it mid-flight
const victim = postgres(DBURL, { max: 1, connection: { statement_timeout: '1' } })
try { await victim`SELECT 1 AS n` } catch (e) { console.log('caller query rejected with', e.code) }
await new Promise((r) => setTimeout(r, 500))
console.log('unhandled rejections:', unhandled.length)
unhandled.forEach((r) => console.log(String(r.stack).split('\n').slice(0, 3).join('\n')))
await victim.end({ timeout: 3 }).catch(() => {})
// cleanup in batches - DROP SCHEMA CASCADE over thousands of types exceeds max_locks_per_transaction
for (;;) {
const names = await setup`
SELECT t.typname FROM pg_catalog.pg_type t
JOIN pg_catalog.pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = ${SCHEMA} AND t.typtype = 'c' LIMIT 500`
if (!names.length) break
await setup.unsafe(names.map(({ typname }) => `DROP TYPE ${SCHEMA}."${typname}" CASCADE;`).join(''))
}
await setup.unsafe(`DROP SCHEMA IF EXISTS ${SCHEMA} CASCADE`)
await setup.end({ timeout: 5 })
Output on 3.4.9 (postgres 16, Node 26):
array-types query: 5.0ms (6000 helper types)
caller query rejected with 57014
unhandled rejections: 1
PostgresError: canceling statement due to statement timeout
at ErrorResponse (.../postgres/src/connection.js:815:30)
at handle (.../postgres/src/connection.js:489:6)
Run the same script with node --unhandled-rejections=throw and the process dies.
Suggested fix
Give the dropped promise a handler and route the failure through the connection's normal error path:
async function fetchArrayTypes() {
needsTypes = false
- const types = await new Query([`
- ...
- `], [], execute)
- types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
+ try {
+ const types = await new Query([`
+ ...
+ `], [], execute)
+ types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
+ } catch (err) {
+ errored(err)
+ }
}
Verified against the repro above, same machine and database, only the driver differing:
| unhandled rejections | caller query | array parsing | |
|---|---|---|---|
| 3.4.9 as published | 1 (fatal under --unhandled-rejections=throw) |
rejects with 57014 |
fine |
| 3.4.9 + diff above | 0 | rejects with 57014 |
fine (int[], text[] round-trip unchanged) |
We are running this as a local pnpm patch with the repro wired up as a regression test, and will drop it as soon as a release handles the cancellation path.
Happy to send this as a PR if the approach looks right - and to cover fetchState() and the unawaited auth handlers in the same pass.
Environment
- postgres 3.4.9
- Node 26.7.0 locally; Vercel's Node serverless runtime in production
- PostgreSQL 16 locally; Supabase/Supavisor in production
Same class of unhandled rejection as #252 and #162.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the deterministic reproduction in the issue, then inspect src/connection.js around ReadyForQuery, fetchArrayTypes(), fetchState(), and the unawaited auth handlers. Done means cancellation produces no unhandled rejection, the caller still receives SQLSTATE 57014, and array parsing remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs, postgresql
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100