hapostgres / hapostgres/pg_auto_failover

pgsql_retry_open_connection() leaks the PGconn from the timed-out attempt, accumulating orphaned idle backends on the monitor until max_connections is exhausted

Open Beginner friendly
#1,178 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
1.4k
Forks
142
Avg merge
5h 8m
Merged PRs (30d)
1

Description

`pgsql_retry_open_connection()` overwrites `pgsql->connection` with a fresh `PQconnectdb()` result without calling `PQfinish()` on the object it replaces — both the failed initial attempt made by `pgsql_open_connection()` and, on every subsequent iteration, the previous failed retry. When the client hits the hard-coded 2-second PGCONNECT_TIMEOUT after the server has already completed authentication — rather than earlier, during the TCP or TLS phase — the abandoned connection is left ESTABLISHED and idle on the monitor. PostgreSQL applies no timeout to an authenticated idle connection (idle_session_timeout is off by default and only exists since PG14), so that backend stays there permanently. The client-side PGconn is leaked as well (fd + memory), which libpq's documentation explicitly forbids. We observed this on a production monitor accumulating at ~2–3 orphaned backends per hour while the keeper's node sat at ~74% CPU steal, reaching 189 orphans against max_connections = 200. Accumulation stopped dead the moment the hypervisor's oversubscription eased and steal dropped to ~6%.

```c
src/bin/common/pgsql.c, pgsql_open_connection() (L517):

/* Make a connection to the database */
pgsql->connection = PQconnectdb(pgsql->connectionString); // L549

if (PQstatus(pgsql->connection) != CONNECTION_OK)
{
if (pgsql->retryPolicy.maxR == 0)
{
...
pgsql_finish(pgsql); // <- finishes here
return NULL;
}

if (!pgsql_retry_open_connection(pgsql)) // <- but NOT here
{ ... }
}
```

Contributor guide

Open the contributing guide

Research direction

Start in src/bin/common/pgsql.c at pgsql_open_connection() and pgsql_retry_open_connection(), following how each PQconnectdb() result replaces pgsql->connection. Confirm the failed connection object is handled before replacement, then verify the retry path no longer leaves authenticated idle PostgreSQL backends or leaked libpq resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.