porsager / porsager/postgres

Copy stream hangs if using gzip stream compression

Open
#499 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
8.7k
Forks
374
Avg merge
11d 16h
Merged PRs (30d)
1

Description

Hello, we caught a cool bug )

Streams work fine when copying, but if the stream is sent to the compression pipe, the entire pool of connections freezes.

Example, playback:

import postgres from 'postgres';
import { createGzip } from 'zlib';

const db = postgres({
  max: 2,
});

for (let i = 0; i < 10; i++) {
  let stream = await db
    .unsafe(`COPY pg_catalog.pg_attribute TO STDOUT`)
    .readable();

  stream.pipe(createGzip()).on('data', console.log);

  await db.unsafe(`SELECT $1 AS "CONTINUE"`, [i]);
}

console.log(await db.unsafe(`SELECT 'DONE' AS "END"`));

I play with pipe, but in any case, something is wrong with the connection pool after sending the stream to the pipe.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the provided script with max: 2, COPY ... TO STDOUT, readable(), and createGzip(), then verify whether the following SELECT calls complete. Inspect the readable stream and connection-pool handling around COPY; done means compressed streams no longer freeze the pool and the later SELECT queries return normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs, postgresql
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.