MagicStack / MagicStack/asyncpg

TimeoutError when calling asyncpg.create_pool()

オープン
#824 コメント 12 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

主要言語
Python
スター
8.1k
フォーク
468
PR マージ指標
30日以内にマージされた PR はありません

説明

  • asyncpg version: 0.24.0
  • PostgreSQL version: 12
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : no SaaS
  • Python version: 3.9.7
  • Platform: Linux/Docker (image python:3.9-slim)
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes
  • If you built asyncpg locally, which version of Cython did you use?: n/a
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : Issue happens with both, though the below error is with uvloop

DISCLAIMER: I am relatively new to async methodologies in Python (I am primary a C# engineer), so I may be doing something completely wrong. Feel free to point anything out! Thats how we learn right? haha

I seem to get the below exception a few times an hour. I am relatively certain its not any kind of connection exhaustion as I am using async with statements like those in documentation, and other apps that use this DB are unaffected, this is the 1st app to use asyncpg.

Stack Trace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 586, in _create_ssl_connection
    tr, pr = await loop.create_connection(
  File "uvloop/loop.pyx", line 2014, in create_connection
  File "uvloop/loop.pyx", line 2001, in uvloop.loop.Loop.create_connection
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 492, in wait_for
    fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/./scripts/scheduler_ahs.py", line 28, in process_ah
    ah_processor = await AHProcessor.create(ah_id)
  File "/app/scripts/process_ah.py", line 39, in create
    db = await Database.create()
  File "/app/lib/db.py", line 13, in create
    pool = await asyncpg.create_pool(
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 407, in _async__init__
    await self._initialize()
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 445, in _initialize
    await asyncio.gather(*connect_tasks)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 127, in connect
    self._con = await self._pool._get_new_connection()
  File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 491, in _get_new_connection
    con = await connect_utils._connect_addr(
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 676, in _connect_addr
    return await __connect_addr(params, timeout, True, *args)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 720, in __connect_addr
    tr, pr = await compat.wait_for(connector, timeout=timeout)
  File "/usr/local/lib/python3.9/site-packages/asyncpg/compat.py", line 66, in wait_for
    return await asyncio.wait_for(fut, timeout)
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 494, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

I am using a DB wrapper class. In my actual app I create an instance of the below class and then call the close_pool() method when my app is done using the database.

class Database:
    def __init__(self, pool: asyncpg.Pool) -> None:
        self._pool = pool

    @classmethod
    async def create(cls, num_connections=3):
        pool = await asyncpg.create_pool(
            dsn=my_conn_string,
            server_settings={"search_path": my_schema},
            min_size=num_connections,
            max_size=num_connections,
        )
        return cls(pool)

    async def fetch(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.fetch(sql, *args)

    async def fetch_as_dataframe(self, sql, *args):
        async with self._pool.acquire() as connection:
            stmt = await connection.prepare(sql)
            columns = [a.name for a in stmt.get_attributes()]
            data = await stmt.fetch(*args)
            return pd.DataFrame(data, columns=columns)

    async def fetchrow(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.fetchrow(sql, *args)

    async def execute(self, sql, *args):
        async with self._pool.acquire() as connection:
            return await connection.execute(sql, *args)

    async def copy_records_to_table(self, table, records):
        async with self._pool.acquire() as connection:
            return await connection.copy_records_to_table(table, records=records)

    async def close_pool(self):
        await self._pool.close()

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

asyncpg.create_pool() で報告されたスタックトレースと、提供されたバージョン、プラットフォーム、プール設定から始めます。Python 3.9、PostgreSQL 12、および asyncio と uvloop の両方でタイムアウトを再現し、その後、asyncpg/connect_utils.py と asyncpg/pool.py に示されている接続セットアップの経路を調査します。再現可能な原因を特定し、タイムアウトの動作を文書化または修正できれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
postgresql, python
領域
databases
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
説明が足りない
初心者へのやさしさ
30/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。