MagicStack / MagicStack/asyncpg

TimeoutError when calling asyncpg.create_pool()

Ouverte
#824 12 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Langage dominant
Python
Étoiles
8.1k
Forks
468
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

  • asyncpg version: 0.24.0
  • PostgreSQL version: 12
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : no SaaS
  • Python version: 3.9.7
  • Platform: Linux/Docker (image python:3.9-slim)
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes
  • If you built asyncpg locally, which version of Cython did you use?: n/a
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : Issue happens with both, though the below error is with uvloop

DISCLAIMER: I am relatively new to async methodologies in Python (I am primary a C# engineer), so I may be doing something completely wrong. Feel free to point anything out! Thats how we learn right? haha

I seem to get the below exception a few times an hour. I am relatively certain its not any kind of connection exhaustion as I am using async with statements like those in documentation, and other apps that use this DB are unaffected, this is the 1st app to use asyncpg.

Stack Trace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 586, in _create_ssl_connection
    tr, pr = await loop.create_connection(
  File "uvloop/loop.pyx", line 2014, in create_connection
  File "uvloop/loop.pyx", line 2001, in uvloop.loop.Loop.create_connection
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 492, in wait_for
    fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/./scripts/scheduler_ahs.py", line 28, in process_ah
    ah_processor = await AHProcessor.create(ah_id)
  File "/app/scripts/process_ah.py", line 39, in create
    db = await Database.create()
  File "/app/lib/db.py", line 13, in create
    pool = await asyncpg.create_pool(
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 407, in _async__init__
    await self._initialize()
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 445, in _initialize
    await asyncio.gather(*connect_tasks)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 127, in connect
    self._con = await self._pool._get_new_connection()
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 491, in _get_new_connection
    con = await connect_utils._connect_addr(
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 676, in _connect_addr
    return await __connect_addr(params, timeout, True, *args)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 720, in __connect_addr
    tr, pr = await compat.wait_for(connector, timeout=timeout)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/compat.py", line 66, in wait_for
    return await asyncio.wait_for(fut, timeout)
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 494, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

I am using a DB wrapper class. In my actual app I create an instance of the below class and then call the close_pool() method when my app is done using the database.

class Database:
    def __init__(self, pool: asyncpg.Pool) -> None:
        self._pool = pool

    @classmethod
    async def create(cls, num_connections=3):
        pool = await asyncpg.create_pool(
            dsn=my_conn_string,
            server_settings={"search_path": my_schema},
            min_size=num_connections,
            max_size=num_connections,
        )
        return cls(pool)

    async def fetch(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.fetch(sql, *args)

    async def fetch_as_dataframe(self, sql, *args):
        async with self._pool.acquire() as connection:
            stmt = await connection.prepare(sql)
            columns = [a.name for a in stmt.get_attributes()]
            data = await stmt.fetch(*args)
            return pd.DataFrame(data, columns=columns)

    async def fetchrow(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.fetchrow(sql, *args)

    async def execute(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.execute(sql, *args)

    async def copy_records_to_table(self, table, records):
        async with self._pool.acquire() as connection:
            return await connection.copy_records_to_table(table, records=records)

    async def close_pool(self):
        await self._pool.close()

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par la stack trace signalée dans asyncpg.create_pool() ainsi que par les versions, la plateforme et les paramètres du pool fournis. Reproduisez le timeout avec Python 3.9, PostgreSQL 12, ainsi qu’avec asyncio et uvloop, puis examinez le chemin de configuration de la connexion indiqué dans asyncpg/connect_utils.py et asyncpg/pool.py. La tâche est terminée lorsqu’une cause reproductible a été identifiée et que le comportement du timeout a été documenté ou corrigé.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
postgresql, python
Domaine
databases
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
30/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.