Using pgxpool connection to query by joining multiple tables does not return all the rows.
- Dominant language
- Go
- Stars
- 14.3k
- Forks
- 1.1k
- Avg merge
- 6d 9h
- Merged PRs (30d)
- 11
Description
**Describe the bug**
Using pgxpool connection to query by joining multiple tables does not return all the rows.
The query does a Join of. multiple tables (5 tables), please see below for a sample query. This was working fine when the number of entries returned was less than 50. But, when the number of entries went above 60, it was returning only 56 entries. Most of the times it returned 56, but sometimes when i change the number of connections, i have see it return 51 entries as well.
Running the exact same query in the DB console returns the right number of entries.
Also, if i use pgx.Connect and do a query based on this connection, it returns the correct number of entries. Only pgxpool has the problem.
I also followed the sample code given in a similar issue:
https://github.com/jackc/pgx/issues/639
I still see the same issue.
I am using version 4. I also upgraded to V5 and that also did not solve the problem. I dumped the cdbpool stats, hoping to see some counts going up, but the values were 0 for stats.CanceledAcquireCount(), stats.MaxIdleDestroyCount(), stats.MaxLifetimeDestroyCount().
**To Reproduce**
Steps to reproduce the behavior:
```go
package main
import (
"context"
"log"
"os"
"github.com/jackc/pgx/v4"
)
func main() {
config, err := pgxpool.ParseConfig(url)
if err != nil {
...
return nil, err
}
// Even tried changing the below to values like 8 or 16
numThreads = runtime.GOMAXPROCS(0)
config.MaxConns = int32(numThreads)
pool, err := pgxpool.ConnectConfig(ctx, config)
....
}
func query(ctx context.Context, tbl string, args ...interface{}) {
conn, err := cdbpool.Acquire(ctx)
// Check for errors
rows, err := conn.Query(ctx, q, args...)
// check for errors
}
The args passed is a join of multiple tables that looks something like
SELECT f1, f2, entry.*, d AS f, tbl1 INNER JOIN tbl2 ON conditions INNER JOIN tbl3 ON cond INNER JOIN tbl4 ON cond INNER JOIN tbl5 cond INNER JOIN cond;
Please note that the above query is just to give an idea. Pasting the exact query in the DB console gives the right results.
```
**Version**
- Go: 1.19
- PostgreSQL: CockroachDB CCL v21.2.17
- pgx: github.com/jackc/pgx/v4 v4.18.1
Contributor guide
Research direction
Start at the pgxpool.Acquire and conn.Query calls in the provided Go reproduction, then compare their row handling with the same query through pgx.Connect. A useful investigation needs the exact five-table query, complete error and rows-iteration checks, and a reproducible CockroachDB setup; done means identifying why pooled queries omit rows and demonstrating the correct result with a focused regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, postgresql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100