MagicStack / MagicStack/asyncpg

Unable to connect to RDS Proxy

未关闭
#952 24 条评论 14 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
Python
星标
8.1k
派生
468
PR 合并指标
30 天内没有已合并 PR

描述

asyncpg=0.26.0
postgres=13.4

We're trying to connect to RDS through and RDS proxy with IAM auth and it doesn't work. We've verified that the provided code works fine when connecting straight to the database with IAM auth. We've also tested that we can connect through the proxy to the database. The only piece that does not work is asyncpg. This also works when using aiopg.

EDIT:
I suspect that it's something with SSL but we've gotten nowhere pretty fast on it. The logs on RDS Proxy don't say anything other than Internal Error which is very.... unhelpful.

import asyncio
import aiopg
import asyncpg
import boto3
import os
import sys
import ssl
import certifi
from urllib.parse import quote_plus


ENDPOINT="<our proxy url>"
PORT="5432"
USER="<our user>"
REGION="us-east-1"
DBNAME="<our db>"

session = boto3.Session(profile_name='profile')
client = session.client('rds')
token = client.generate_db_auth_token(DBHostname=ENDPOINT, Port=PORT, DBUsername=USER, Region=REGION) # type: ignore


async def main():
    try:
        print("trying connection")
        conn = await asyncpg.connect(dsn=f"postgres://{USER}:{quote_plus(token)}@{ENDPOINT}:5432/{DBNAME}?sslmode=require&sslrootcert=./AmazonRootCA1.pem")
        #conn = await asyncpg.connect(user=USER, password=quote_plus(token), database=DBNAME, host=ENDPOINT, ssl='require')
        print("connected, trying query")
        print(await conn.fetch("SELECT 'connected'"))
    except Exception as e:
        print("CAUSE", e.__cause__)
        raise

    # async with aiopg.connect(f'dbname={DBNAME} user={USER} password={token} host={ENDPOINT} sslmode=require') as conn:
    #     async with conn.cursor() as cur:
    #         await cur.execute("SELECT 'connected'")

    #         async for row in cur:
    #             print(row)

asyncio.run(main())

The error we get back is

Traceback (most recent call last):
  File "/tmp/test/__init__.py", line 50, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/tmp/test/__init__.py", line 35, in main
    conn = await asyncpg.connect(dsn=f"postgres://{USER}:{quote_plus(token)}@{ENDPOINT}:5432/{DBNAME}?sslmode=require&sslrootcert=./AmazonRootCA1.pem")
  File "/home/charles/.local/lib/python3.10/site-packages/asyncpg/connection.py", line 2093, in connect
    return await connect_utils._connect(
  File "/home/charles/.local/lib/python3.10/site-packages/asyncpg/connect_utils.py", line 889, in _connect
    return await _connect_addr(
  File "/home/charles/.local/lib/python3.10/site-packages/asyncpg/connect_utils.py", line 776, in _connect_addr
    return await __connect_addr(params, timeout, False, *args)
  File "/home/charles/.local/lib/python3.10/site-packages/asyncpg/connect_utils.py", line 839, in __connect_addr
    await compat.wait_for(connected, timeout=timeout)
  File "/home/charles/.local/lib/python3.10/site-packages/asyncpg/compat.py", line 66, in wait_for
    return await asyncio.wait_for(fut, timeout)
  File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
    return fut.result()
asyncpg.exceptions.ConnectionDoesNotExistError: connection was closed in the middle of operation```

贡献指南

这个仓库没有索引到贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先,使用 asyncpg 0.26.0 针对 RDS Proxy 运行提供的 Python 复现程序,然后跟踪经过 asyncpg.connect 和 asyncpg.connect_utils._connect 的连接路径。将其经过 IAM 身份验证的 SSL 连接与正常工作的直接 PostgreSQL 和 aiopg 连接进行比较。完成的标准是确定 asyncpg 是否可以通过代理进行连接,并记录或修正失败行为。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, postgresql, python
领域
databases
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。