payloadcms / payloadcms/payload

vercelPostgresAdapter fails on large queries (68KB+ SQL with 30+ lateral joins)

Open
#16,256 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
TypeScript
Stars
44.8k
Forks
4.2k
Avg merge
2d 21h
Merged PRs (30d)
53

Description

Description

@payloadcms/db-vercel-postgres fails to execute large SQL queries on Vercel when using NeonDB. The same queries work fine locally and through raw pg.Pool.

Our pages collection has 31 block types in a layout blocks field. Payload + Drizzle generates a ~68KB SQL statement with 30+ lateral joins to query this collection. This query:

  • ✅ Executes in 259ms on NeonDB dashboard (simple query protocol)
  • ✅ Executes in 177ms through raw pg.Pool (TCP, named prepared statement)
  • ✅ Works locally with postgresAdapter (pg.Pool over TCP)
  • ✅ Works locally with vercelPostgresAdapter (same connection string)
  • Fails on Vercel with vercelPostgresAdapter — times out waiting for NeonDB response, even with 60s function timeout

Switching from vercelPostgresAdapter to postgresAdapter on Vercel immediately fixes the issue.

Root Cause

vercelPostgresAdapter uses @vercel/postgres@neondatabase/serverless, which sends queries over WebSocket rather than TCP. The 68KB SQL payload appears to hit protocol-level limits or buffering issues in the WebSocket transport that TCP handles without problems.

Tested with both pooled (-pooler hostname) and unpooled (direct) NeonDB connections — both fail on Vercel with vercelPostgresAdapter.

Secondary Issue: Connection Pool Leak in connectWithReconnect

While investigating, we also found that connectWithReconnect in both @payloadcms/db-postgres and @payloadcms/db-vercel-postgres calls pool.connect() to check out a client but never calls result.release() to return it to the pool:

// connect.js (db-postgres v3.81.0, line 4-37)
const connectWithReconnect = async function({ adapter, pool, reconnect = false }) {
    let result;
    if (!reconnect) {
        result = await pool.connect(); // ← checks out a client
    }
    // ...
    result.prependListener('error', (err) => { /* ... */ });
    // ← result.release() is NEVER called
};

This permanently holds one connection from the pool. With the default max: 5, this leaves only 4 connections for actual queries — exacerbating pool exhaustion when Payload runs multiple concurrent queries per find() call (selectDistinct + findMany + countDistinct, multiplied by depth > 0 relationship population).

Environment

  • Payload: 3.81.0
  • @payloadcms/db-vercel-postgres: 3.79.1
  • @payloadcms/db-postgres: 3.81.0
  • Drizzle ORM: 0.44.7
  • Next.js: 15.4.10
  • Database: NeonDB (serverless PostgreSQL, eu-central-1)
  • Hosting: Vercel (Node.js runtime, not edge)

Workaround

Use postgresAdapter (from @payloadcms/db-postgres) instead of vercelPostgresAdapter on Vercel. This uses pg.Pool with TCP connections, which handles large queries without issues. vercelPostgresAdapter is only required for Edge Runtime.

Also increase pool.max from the default — with 31 block types and depth > 0, Payload can generate many concurrent sub-queries that exhaust a small pool, especially with the leaked connection from connectWithReconnect.

// payload.config.ts
import { postgresAdapter } from '@payloadcms/db-postgres'

db: postgresAdapter({
  pool: {
    connectionString: process.env.POSTGRES_URL || '',
    max: 20, // default is too low for collections with many blocks
  },
  push: false,
})

Steps to Reproduce

  1. Create a pages collection with 20+ block types in a layout blocks field
  2. Configure vercelPostgresAdapter with NeonDB
  3. Deploy to Vercel (Node.js runtime)
  4. Request a page — the query times out

Expected: Query completes successfully (as it does with postgresAdapter)
Actual: Connection to NeonDB times out via WebSocket transport

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with connect.js in @payloadcms/db-postgres and @payloadcms/db-vercel-postgres, then reproduce the large-query timeout using the Vercel Postgres adapter and NeonDB setup described here. Check the connection handling and WebSocket path against the working pg.Pool path. Done means large queries complete on Vercel and checked-out pool clients are returned without regressing normal queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.