porsager / porsager/postgres

fetch_types: false, deadlocks reserve() and alternatively pulls ~10KB from server before any work and can burn GBs of egress per month

Open
#1,219 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
8.7k
Forks
374
Avg merge
11d 16h
Merged PRs (30d)
1

Description

Summary

fetch_types makes every new connection pull ~10KB from the server before it does any work, and can burn GBs of egress per month.

The documented remedy, fetch_types: false, deadlocks reserve() — so any consumer that reserves, including kysely-postgres-js, cannot turn it off. The two together are the problem: the cost is invisible until it appears on a bill, and the escape hatch fails silently.

Version: postgres@3.4.9, Node v24.1.0. Not runtime-specific — plain import postgres from 'postgres'.

1. The cost

On a bare select 1 — bytes received on the socket by a pool opening one connection:

fetch_types bytes received
true (default) 10,994
false 686

94% of what a fresh connection reads from the server is the array-type catalog. The query returns 415 rows here, and scales with the array types the server knows — not with anything the application does.

On one production database, a 2-month pg_stat_statements slice:

   calls   |    rows     | query
  448,936  | 181,368,184 | select b.oid, b.typarray from pg_catalog.pg_type a ...

91% of every row the database returned, across all statements. At ~9,800 connections/day (serverless, one pool per invocation) that is ~100 MB/day, ~3 GB/month — against a 5 GB quota, billed as Supavisor pooler egress with no attribution back to a query, so no provider diagnostic points at it.

Each connection in a pool also re-fetches a result the pool already holds: options.shared.typeArrayMap is populated and addArrayType early-returns, but the fetch still runs.

2. The remedy is unusable

fetch_types: false is documented, and it deadlocks reserve() on a cold pool:

cold pool, fetch_types: true  -> reserve() OK
cold pool, fetch_types: false -> reserve() never settled
warm pool, fetch_types: false -> reserve() OK
warm pool, fetch_types: true  -> reserve() OK
import postgres from 'postgres'

const sql = postgres(url, { max: 1, fetch_types: false })
await sql.reserve()   // never settles — no error, no timeout

The promise stays pending forever. Plain queries work fine with fetch_types: false, which is what makes this easy to miss: the option looks like it works until something reserves. kysely-postgres-js acquires every connection via sql.reserve(), so the entire query path hangs — in our case a full integration suite went from 2s to 17/17 timeouts, with no error to point at the cause.

Running one throwaway query before anything reserves works around it (#1203), but that depends on pool state rather than fixing anything.

Where it comes from

ReadyForQuery in src/connection.js:

if (needsTypes) {
  initial.reserve && (initial = null)
  return fetchArrayTypes()
}

initial && !initial.reserve && execute(initial)
options.shared.retries = retries = 0
initial = null
return

With fetch_types: true, fetchArrayTypes() forces a second ReadyForQuery in which initial is already null. Control then falls through to the tail of the same function:

connection.reserved
  ? ... : connection.reserved()
  : ending ? terminate() : onopen(connection)

and onopen (src/index.js) resolves the waiter via if (query.reserve) return query.reserve(c).

With fetch_types: false there is no second round trip. execute(initial) is skipped because initial.reserve is set, initial is discarded, and the function returns before reaching onopen — so nothing ever resolves the reserve. The type-fetch round trip is accidentally load-bearing for reserve().

I have not tested a patch, but the asymmetry looks like the thing to fix: the needsTypes branch already nulls initial for a reserve and lets the next ReadyForQuery resolve it, while the non-fetching path returns early instead of falling through to onopen.

Related
  • #1203 — same reserve() hang, reported as Cloudflare/workerd-specific and attributed to the cloudflare:sockets polyfill making connection setup async. The matrix above shows it reproduces on plain Node and that the trigger is fetch_types, not the runtime — that report always sets fetch_types: false, so the variable was never isolated. Likely the same root cause.
  • #1195 — a different reserve() stranding (pooled connection terminated server-side while the reserve is queued).
  • #1136 (closed) — the same pg_type query, reported for missing CommandComplete.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the cold-pool hang with fetch_types: false and reserve(), then read the ReadyForQuery handling in src/connection.js and the reserve resolution in src/index.js. Compare the fetch_types true and false paths and add regression coverage for reserve() without the type fetch. Done means reserve() settles on a cold pool while the documented option avoids the unnecessary type query.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs, postgresql
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.