MagicStack / MagicStack/asyncpg

Continuous memory increase in Sanic web server after calling `loop.start_tls(...)` in `connect_utils._create_ssl_connection`

Abierto
#1,004 2 comentarios 0 reacciones 1 asignado Ver en GitHub

@fantix ya está trabajando en esto.

Desde el 17/2/2023.

Lenguaje dominante
Python
Estrellas
8.1k
Forks
468
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

* **asyncpg version**: `0.2.7`
* **PostgreSQL version**: `PostgreSQL 13.3 (Debian 13.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit`
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**: I use PostgreSQL installed in docker image `kartoza/postgis:pr-333-13-3.1`
* **Python version**: `3.8.10`
* **Platform**: `20.04.1-Ubuntu x86_64`
* **Do you use pgbouncer?**: Not sure, I use it indirectly. (behind `sqlalchemy`)
* **Did you install asyncpg with pip?**: Yes
* **If you built asyncpg locally, which version of Cython did you use?**: /
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: I see the code below is based on uvloop.

**Background**: I built a simple web server using `sqlalchemy` and `sanic`, then created a request handler function that can query my local PostgreSQL server running in docker. However, I saw continuous memory increase while benching my handler function.

After running `tracemalloc` for help, I got the result below:
```bash
Top 10 lines
#1: /home/tusimple/Workspace/projects/map-workdata-manager/venv/lib/python3.8/site-packages/asyncpg/connect_utils.py:700: 5670.1 KiB
new_tr = await loop.start_tls(
```

The code for creating the async_engine is:
```python
from sqlalchemy.ext.asyncio import create_async_engine
engine = create_async_engine("postgresql+asyncpg://{}:{}@{}/{}".format(...))
```

At first, I tried adding `?ssl=disable` after the previous connect string in order to prevent `connect_utils._create_ssl_connection` calls, and it works.
Then, I tried to make some stupid changes to the source code of `asyncpg`, because I think this SSL transport is not closed after creating new transport. (I'm not sure is that correct)
```python
# line 697
if hasattr(loop, 'start_tls'):
if do_ssl_upgrade:
try:
new_tr = await loop.start_tls(
tr, pr, ssl_context, server_hostname=host) # <--- here
except (Exception, asyncio.CancelledError):
tr.close()
raise
else:
new_tr = tr
# line 706
```
Finally, I modified the code starts from line 702 to:
```python
except (Exception, asyncio.CancelledError):
raise
finally:
tr.close()
```
After this modification, my web server keeps working fine, but I don't know whether this is a proper modification.

If you have any questions or ideas, please let me know, thank you so much!

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.