MagicStack / MagicStack/asyncpg
How are Postgres server restarts handled?
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
Hi guys,
We are using asyncpg in a web application.
Upon starting the web server, we do the following two things:
1. `db = asyncpg.create_pool` to create the connection pool for the application's DB connections.
1. `conn = await db.pool.acquire(); await conn.add_listener('foo', _callback)` to register long-lived listeners to catch and handle `NOTIFY` events from the DB.
Recently I restarted the Postgres server and noticed something interesting. First, connections to the DB using the `db` pool object saw no interruption and were still able to connect to the DB server despite it having a new PID (since the process had been restarted).
However, the `conn` the application relied on to receive async notifications from the server was dead/closed and so the application logic dependent upon receiving those notifications was not functioning. (This is how I reproduced this issue and discovered the cause to have been a PG server restart.)
* Why is the pool unaffected by restart? It makes sense the connection instance is dead since the socket is closed after restarting the server, but I'm a bit surprised the pool is not affected.
* I suppose the best strategy for checking for dead DB connections would be to simply check `conn.is_closed` periodically - or, is there some "on connection close" callback I can define to retry/reopen a connection (since obtaining new connections from the pool still works after restart)?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the asyncpg.create_pool, pool.acquire, add_listener, and conn.is_closed entry points described in the report, along with PostgreSQL NOTIFY behavior across a server restart. Determine how pooled connections and long-lived listeners are expected to behave, and document the supported way to detect and recover closed listener connections.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100