hapostgres / hapostgres/pg_auto_failover
pgsql_retry_open_connection() leaks the PGconn from the timed-out attempt, accumulating orphaned idle backends on the monitor until max_connections is exhausted
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 142
- Avg merge
- 5h 8m
- Merged PRs (30d)
- 1
Description
`pgsql_retry_open_connection()` overwrites `pgsql->connection` with a fresh `PQconnectdb()` result without calling `PQfinish()` on the object it replaces — both the failed initial attempt made by `pgsql_open_connection()` and, on every subsequent iteration, the previous failed retry. When the client hits the hard-coded 2-second PGCONNECT_TIMEOUT after the server has already completed authentication — rather than earlier, during the TCP or TLS phase — the abandoned connection is left ESTABLISHED and idle on the monitor. PostgreSQL applies no timeout to an authenticated idle connection (idle_session_timeout is off by default and only exists since PG14), so that backend stays there permanently. The client-side PGconn is leaked as well (fd + memory), which libpq's documentation explicitly forbids. We observed this on a production monitor accumulating at ~2–3 orphaned backends per hour while the keeper's node sat at ~74% CPU steal, reaching 189 orphans against max_connections = 200. Accumulation stopped dead the moment the hypervisor's oversubscription eased and steal dropped to ~6%.
```c
src/bin/common/pgsql.c, pgsql_open_connection() (L517):
/* Make a connection to the database */
pgsql->connection = PQconnectdb(pgsql->connectionString); // L549
if (PQstatus(pgsql->connection) != CONNECTION_OK)
{
if (pgsql->retryPolicy.maxR == 0)
{
...
pgsql_finish(pgsql); // <- finishes here
return NULL;
}
if (!pgsql_retry_open_connection(pgsql)) // <- but NOT here
{ ... }
}
```
Contributor guide
Research direction
Start in src/bin/common/pgsql.c at pgsql_open_connection() and pgsql_retry_open_connection(), following how each PQconnectdb() result replaces pgsql->connection. Confirm the failed connection object is handled before replacement, then verify the retry path no longer leaves authenticated idle PostgreSQL backends or leaked libpq resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, postgresql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100