MagicStack / MagicStack/asyncpg

Making pool connection cycling async

Open
#255 5 comments 5 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

* **asyncpg version**: 0.14
* **PostgreSQL version**: 9.5/9.6
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**: AWS RDS
* **Python version**: 3.6
* **Platform**: linux in docker
* **Do you use pgbouncer?**: Not in this instances
* **Did you install asyncpg with pip?**: yes
* **If you built asyncpg locally, which version of Cython did you use?**:
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: We only use asyncio

We have some application containers that use asyncpg connection pools, every now and again my performance tracing on our application request handlers shows that the `connect` method in the `Pool` takes a significant amount of time (sometimes 150ms) when establishing a connection to our RDS postgres instance (same Region).

The Pool has options to control how often a connection is pro-actively cycled, but my concern is that this results in clients of the pool having to wait for a connection to be established - I would much rather push this into a background async task who's job it is to take connections in/out of service so that inbound http requests to our server never incur the latency of performing the connection themselves.

Is this already doable or would it require some changes to asyncpg?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the Pool.connect entry point and the existing pool options for proactively cycling connections. Trace when connection establishment occurs for inbound requests and determine what changes would be needed for background cycling; done means pool clients no longer incur that connection-establishment latency.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.