MagicStack / MagicStack/asyncpg
Making pool connection cycling async
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
- asyncpg version: 0.14
- PostgreSQL version: 9.5/9.6
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: AWS RDS - Python version: 3.6
- Platform: linux in docker
- Do you use pgbouncer?: Not in this instances
- Did you install asyncpg with pip?: yes
- If you built asyncpg locally, which version of Cython did you use?:
- Can the issue be reproduced under both asyncio and
uvloop?: We only use asyncio
We have some application containers that use asyncpg connection pools, every now and again my performance tracing on our application request handlers shows that the connect method in the Pool takes a significant amount of time (sometimes 150ms) when establishing a connection to our RDS postgres instance (same Region).
The Pool has options to control how often a connection is pro-actively cycled, but my concern is that this results in clients of the pool having to wait for a connection to be established - I would much rather push this into a background async task who's job it is to take connections in/out of service so that inbound http requests to our server never incur the latency of performing the connection themselves.
Is this already doable or would it require some changes to asyncpg?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the Pool.connect entry point and the existing pool options for proactively cycling connections. Trace when connection establishment occurs for inbound requests and determine what changes would be needed for background cycling; done means pool clients no longer incur that connection-establishment latency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100