Intermittent undefined values in query results
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 8.7k
- Forks
- 374
- Avg merge
- 11d 16h
- Merged PRs (30d)
- 1
Description
Hi everyone! I've encountered an intermittent and unexpected behaviour with postgres.js.
Summary
The result from postgres.js queries occasionally contains undefined values. The result in such cases would look like:
[{ id: 1 }, { id: 2 }, undefined]
It's not clear to me if this is an expected behaviour and, if so, under what circumstances it occurs.
This unexpected behaviour leads to errors like: Cannot read properties of undefined (reading 'id') as until recently, our results were always predictable: we'd receive either an empty or non-sparse array as a result.
Details
The issue appears sporadically and doesn't seem to correlate directly with any specific type of query. However, we've observed increased Postgres latency, with some queries timing out around the same period. It's unclear if these timeouts are somehow related to this behaviour; the frequency of timeouts is much higher compared to the undefined values.
I am unable to reproduce the issue locally; we only have production logs showing this occurring a few times a day.
Here is a portion from our codebase:
this.sqlClient = postgres({ ... });
...
const exec = this.sqlClient.unsafe(query.text, query.values, { prepare: true });
const timeoutMs = opts.timeoutMs ?? this.defaultQueryTimeoutMs;
if (timeoutMs) {
const timer = setTimeout(() => exec.cancel(), timeoutMs + 1);
return exec.finally(() => clearTimeout(timer));
} else {
return exec;
}
Workaround
If I apply a filter to the results returned by exec, as shown in the example below, it effectively prevents the errors we've been experiencing. This leads me to believe that the undefined values are indeed originating from postgres.js, rather than from other parts of our stack.
exec.then((results) => results.filter((result) => result !== undefined))
Environment
| Component | Version |
|---|---|
| postgres.js | 3.4.3 |
| PostgreSQL | 15.3 |
| Node.js | 20.5.1 |
Happy to provide more details. Any insights or guidance would be greatly appreciated.
Thank you!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the postgres.js unsafe query path and the exec.cancel() timeout handling shown in the report. Compare those entry points with the production logs and the reported PostgreSQL, Node.js, and postgres.js versions. Done means identifying whether timeout or cancellation can produce sparse results and documenting or fixing the behavior with evidence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, postgresql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100