porsager / porsager/postgres

Intermittent undefined values in query results

Open
#795 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
8.7k
Forks
374
Avg merge
11d 16h
Merged PRs (30d)
1

Description

Hi everyone! I've encountered an intermittent and unexpected behaviour with postgres.js.

Summary

The result from postgres.js queries occasionally contains undefined values. The result in such cases would look like:

  • [{ id: 1 }, { id: 2 }, undefined]

It's not clear to me if this is an expected behaviour and, if so, under what circumstances it occurs.

This unexpected behaviour leads to errors like: Cannot read properties of undefined (reading 'id') as until recently, our results were always predictable: we'd receive either an empty or non-sparse array as a result.

Details

The issue appears sporadically and doesn't seem to correlate directly with any specific type of query. However, we've observed increased Postgres latency, with some queries timing out around the same period. It's unclear if these timeouts are somehow related to this behaviour; the frequency of timeouts is much higher compared to the undefined values.

I am unable to reproduce the issue locally; we only have production logs showing this occurring a few times a day.

Here is a portion from our codebase:

this.sqlClient = postgres({ ... });

...

const exec = this.sqlClient.unsafe(query.text, query.values, { prepare: true });

const timeoutMs = opts.timeoutMs ?? this.defaultQueryTimeoutMs;

if (timeoutMs) {
    const timer = setTimeout(() => exec.cancel(), timeoutMs + 1);
    return exec.finally(() => clearTimeout(timer));
} else {
    return exec;
}

Workaround

If I apply a filter to the results returned by exec, as shown in the example below, it effectively prevents the errors we've been experiencing. This leads me to believe that the undefined values are indeed originating from postgres.js, rather than from other parts of our stack.

exec.then((results) => results.filter((result) => result !== undefined))

Environment

Component Version
postgres.js 3.4.3
PostgreSQL 15.3
Node.js 20.5.1

Happy to provide more details. Any insights or guidance would be greatly appreciated.

Thank you!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the postgres.js unsafe query path and the exec.cancel() timeout handling shown in the report. Compare those entry points with the production logs and the reported PostgreSQL, Node.js, and postgres.js versions. Done means identifying whether timeout or cancellation can produce sparse results and documenting or fixing the behavior with evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, postgresql
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.