jackc / jackc/pgx

Pgx uses a named prepared statement for a query even when it is not frequently used

Open
#2,308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

Here is a simple reproduction with Postgres and not using any server side poolers.

```
package main

import (
"context"
"fmt"
"os"
"time"

"github.com/jackc/pgx/v5/pgxpool"
)

func main() {
connString := "postgresql://use:pass@host/postgres?sslmode=require"

if os.Getenv("DATABASE_URL") != "" {
connString = os.Getenv("DATABASE_URL")
}

// Connect to the database using pgxpool
config, err := pgxpool.ParseConfig(connString)
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to parse connection string: %v\n", err)
os.Exit(1)
}

pool, err := pgxpool.NewWithConfig(context.Background(), config)
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to create connection pool: %v\n", err)
os.Exit(1)
}
defer pool.Close()

// Begin a transaction
tx, err := pool.Begin(context.Background())
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to begin transaction: %v\n", err)
os.Exit(1)
}
defer tx.Rollback(context.Background()) // Rollback if not committed

// Execute a simple scalar query within the transaction
var count int
err = tx.QueryRow(context.Background(), "SELECT 42").Scan(&count)
if err != nil {
fmt.Fprintf(os.Stderr, "Query failed: %v\n", err)
os.Exit(1)
}

fmt.Printf("Query result: %d\n", count)

// Now let's look at specific columns that we know will work
rows, err := tx.Query(context.Background(), `
SELECT name, statement, prepare_time, from_sql FROM pg_prepared_statements
`)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to query sample data: %v\n", err)
os.Exit(1)
}
defer rows.Close()

fmt.Println("\nSample data from pg_prepared_statements:")
for rows.Next() {
var name, statement string
var prepareTime time.Time
var fromSQL bool

if err := rows.Scan(&name, &statement, &prepareTime, &fromSQL); err != nil {
fmt.Fprintf(os.Stderr, "Failed to scan row: %v\n", err)
os.Exit(1)
}

fmt.Printf(" Name: %s\n", name)
fmt.Printf(" Statement: %s\n", statement)
fmt.Printf(" Prepare Time: %v\n", prepareTime)
fmt.Printf(" From SQL: %v\n", fromSQL)
fmt.Println(" ---")
}

// Query PostgreSQL for the number of prepared statements in the same transaction
// After examining the structure, we'll use a more targeted query with the correct column
var preparedCount int
err = tx.QueryRow(context.Background(), `
SELECT count(*) FROM pg_prepared_statements
WHERE name LIKE 'pgx_%'
`).Scan(&preparedCount)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to query prepared statements: %v\n", err)
os.Exit(1)
}

// Commit the transaction
err = tx.Commit(context.Background())
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to commit transaction: %v\n", err)
os.Exit(1)
}
}
```

This produces the following output

```
Query result: 42

Sample data from pg_prepared_statements:
Name: stmtcache_002286f7369ec82b7ac8b0a5614dbd0269f166f3fd472836
Statement: SELECT 42
Prepare Time: 2025-04-24 12:59:04.092426 -0500 CDT
From SQL: false
---
Name: stmtcache_22e42277957c3fd058a9cae42b4259d6929026086b0fdaec
Statement:
SELECT name, statement, prepare_time, from_sql FROM pg_prepared_statements

Prepare Time: 2025-04-24 12:59:04.15765 -0500 CDT
From SQL: false
---
```

Note that the `SELECT 42` query was prepared as a named prepared statement even when it was used only once. This leads to a large number of named prepared statements, some of which might only be used only once. Poolers like pgbouncer maintains a local cache of prepared statements, this puts pressure on such poolers to maintain a larger list of statements. While it is possible to completely disable statement cache, that will affect all queries negatively.

I think we should only prepare a named statement after it has been used a couple of times, that threshold can be configurable in `ConnConfig`.

Contributor guide

Open the contributing guide

Research direction

Start at ConnConfig and trace the statement-cache behavior exercised by the pgxpool reproduction, especially how the SELECT 42 query becomes a named prepared statement. Define the configurable reuse threshold and verify that statements are named only after reaching it while statement caching remains available.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.