jackc / jackc/pgx

my pool keeps running out of connections

Open
#1,974 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

I'm using pgxpool in a kubernetes based http that connects to a database in AWS RDS. The prod deployment of the app is running pgx/v4 and creates a new connection to the DB for each request which then go through pgbouncer and that has been working fairly smoothly for a few years. We do run into the occasional memory leak issue from connections not closing properly but a quick restart fixes that.

I'm trying once again to use pgxpool using v5 release, but I'm not certain what I'm doing incorrectly as the issue I keep running into is that eventually the pool just runs out of connections and the app just chokes. I've looked through my code to make sure I'm closing rows where needed, have tried to acquire and release connections directly from and back to the pool but still no luck. The pgxpool connections are not going through pgbouncer -- this connects directly to the DB.

I've also tried tweaking various settings such as healthcheck timers, idle connection lifetime settings, etc., but the end result is pretty much the same every time. After a while, the pool connections are exhausted and my pool stats look as follows -- the app does not recover w/out a full restart:

```map[
acquireCount:110079
acquireDuration:7.877843226s
acquiredConns:32
canceledAcquireCount:265
constructingConns:0
emptyAcquireCount:1385
idleConns:0
maxConns:32
maxIdleDestroyCount:0
maxLifetimeDestroyCount:927
newConnsCount:1398
totalConns:32]
```

These are the connection values I'm currently using:
```config.MaxConnIdleTime = 5 * time.Minute
config.MaxConnLifetime = 10 * time.Minute
config.MaxConnLifetimeJitter = 1 * time.Minute
config.HealthCheckPeriod = 1 * time.Minute
config.MaxConns = 32
config.MinConns = 4
```

I'm not sure what I'm doing incorrectly, or if there are any other settings I should tweak here so that the pool doesn't get exhausted.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the exhaustion with the reported pgx/v5 pool settings and inspect the acquire, release, and pool-stat behavior described in the issue. Compare the application’s connection and rows cleanup paths against the pool configuration; done means the pool remains usable without exhausting all 32 connections or requiring a restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.