MagicStack / MagicStack/asyncpg

How are Postgres server restarts handled?

Open
#421 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.1k
Forks
469
Avg merge
18m
Merged PRs (30d)
4

Description

Hi guys,

We are using asyncpg in a web application.

Upon starting the web server, we do the following two things:

  1. db = asyncpg.create_pool to create the connection pool for the application's DB connections.
  2. conn = await db.pool.acquire(); await conn.add_listener('foo', _callback) to register long-lived listeners to catch and handle NOTIFY events from the DB.

Recently I restarted the Postgres server and noticed something interesting. First, connections to the DB using the db pool object saw no interruption and were still able to connect to the DB server despite it having a new PID (since the process had been restarted).

However, the conn the application relied on to receive async notifications from the server was dead/closed and so the application logic dependent upon receiving those notifications was not functioning. (This is how I reproduced this issue and discovered the cause to have been a PG server restart.)

  • Why is the pool unaffected by restart? It makes sense the connection instance is dead since the socket is closed after restarting the server, but I'm a bit surprised the pool is not affected.
  • I suppose the best strategy for checking for dead DB connections would be to simply check conn.is_closed periodically - or, is there some "on connection close" callback I can define to retry/reopen a connection (since obtaining new connections from the pool still works after restart)?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the asyncpg.create_pool, pool.acquire, add_listener, and conn.is_closed entry points described in the report, along with PostgreSQL NOTIFY behavior across a server restart. Determine how pooled connections and long-lived listeners are expected to behave, and document the supported way to detect and recover closed listener connections.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
backend, databases
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.