jackc / jackc/pgx

Using pgxpool connection to query by joining multiple tables does not return all the rows.

Open
#1,834 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

**Describe the bug**
Using pgxpool connection to query by joining multiple tables does not return all the rows.
The query does a Join of. multiple tables (5 tables), please see below for a sample query. This was working fine when the number of entries returned was less than 50. But, when the number of entries went above 60, it was returning only 56 entries. Most of the times it returned 56, but sometimes when i change the number of connections, i have see it return 51 entries as well.
Running the exact same query in the DB console returns the right number of entries.

Also, if i use pgx.Connect and do a query based on this connection, it returns the correct number of entries. Only pgxpool has the problem.

I also followed the sample code given in a similar issue:
https://github.com/jackc/pgx/issues/639

I still see the same issue.

I am using version 4. I also upgraded to V5 and that also did not solve the problem. I dumped the cdbpool stats, hoping to see some counts going up, but the values were 0 for stats.CanceledAcquireCount(), stats.MaxIdleDestroyCount(), stats.MaxLifetimeDestroyCount().

**To Reproduce**
Steps to reproduce the behavior:

```go
package main

import (
"context"
"log"
"os"

"github.com/jackc/pgx/v4"
)

func main() {
config, err := pgxpool.ParseConfig(url)
if err != nil {
...
return nil, err
}

// Even tried changing the below to values like 8 or 16
numThreads = runtime.GOMAXPROCS(0)
config.MaxConns = int32(numThreads)
pool, err := pgxpool.ConnectConfig(ctx, config)
....
}

func query(ctx context.Context, tbl string, args ...interface{}) {
conn, err := cdbpool.Acquire(ctx)
// Check for errors
rows, err := conn.Query(ctx, q, args...)
// check for errors
}

The args passed is a join of multiple tables that looks something like
SELECT f1, f2, entry.*, d AS f, tbl1 INNER JOIN tbl2 ON conditions INNER JOIN tbl3 ON cond INNER JOIN tbl4 ON cond INNER JOIN tbl5 cond INNER JOIN cond;

Please note that the above query is just to give an idea. Pasting the exact query in the DB console gives the right results.
```

**Version**
- Go: 1.19
- PostgreSQL: CockroachDB CCL v21.2.17
- pgx: github.com/jackc/pgx/v4 v4.18.1

Contributor guide

Open the contributing guide

Research direction

Start at the pgxpool.Acquire and conn.Query calls in the provided Go reproduction, then compare their row handling with the same query through pgx.Connect. A useful investigation needs the exact five-table query, complete error and rows-iteration checks, and a reproducible CockroachDB setup; done means identifying why pooled queries omit rows and demonstrating the correct result with a focused regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.