porsager / porsager/postgres

Pipelined query resolves with a sparse result array when the previous query errors mid-stream (`rows` counter not reset on ErrorResponse)

Open Beginner friendly
#1,181 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
8.7k
Forks
374
Avg merge
11d 16h
Merged PRs (30d)
1

Description

Summary

When a query fails while streaming rows (PostgreSQL sends ErrorResponse after some DataRow messages — e.g. statement timeout, cancellation, or a per-row expression error), the connection-scoped rows counter is never reset. The next pipelined query on the same connection then writes its rows starting at that stale offset, and resolves successfully with a sparse array: leading holes at indices 0..k-1, result.length = holes + actual rows, while result.count stays correct.

The sparse result is silent — no error is thrown for the second query — and typically explodes later in application code (e.g. new Map(rows.map(...))TypeError: Iterator value undefined is not an entry object, which is how we found it in production behind drizzle-orm).

Reproduction (postgres.js 3.4.5, any PostgreSQL)

import postgres from 'postgres'

const sql = postgres(process.env.DATABASE_URL, { max: 1 })

// Query A: streams 3 rows, then errors on row 4 (division by zero)
const a = sql`SELECT i, 1/(4-i) AS boom FROM generate_series(1,10) i`.catch(e => e.message)
// Query B: pipelined behind A on the same connection
const b = sql`SELECT x FROM generate_series(1,5) x`

console.log('A:', await a)                    // "division by zero"  ← expected
const rows = await b                          // resolves fine       ← bug below
console.log('length:', rows.length)           // 8   (should be 5)
console.log('count:', rows.count)             // 5   (correct)
console.log('holes:', [...Array(rows.length).keys()].filter(i => !(i in rows)))  // [0, 1, 2]

new Map(rows.map(r => [r.x, r]))              // TypeError: Iterator value undefined is not an entry object

await sql.end()

Observed output:

A: division by zero
length: 8
count: 5
holes: [ 0, 1, 2 ]
TypeError: Iterator value undefined is not an entry object

Suggested fix

Reset the counter when a query errors, e.g. in ErrorResponse (or in errored):

  function ErrorResponse(x) {
+   rows = 0
    query && (query.cursorFn || query.describeFirst) && write(Sync)

(Resetting in ReadyForQuery before activating the next query would also work.)

Environment

  • postgres.js: 3.4.5 (code path unchanged on current master)
  • Node.js: v22 (also reproduced on v24)
  • PostgreSQL: 16 (via PgBouncer in production; reproduced against a direct connection locally)
  • Seen in production behind drizzle-orm (.map() preserves the holes through every layer)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The relevant entry points named are ErrorResponse and errored; start by tracing how the connection-scoped rows counter changes during DataRow streaming and how the next pipelined query is activated. Reproduce the issue with the supplied PostgreSQL and Node.js script, then verify that the follow-up query returns a dense five-row array after the earlier query errors mid-stream.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, postgresql
Domain
database
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.