MagicStack / MagicStack/asyncpg

`TypeError: cannot unpack non-iterable NoneType object` when using docker bridge network

Ouverte
#1,030 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Langage dominant
Python
Étoiles
8.1k
Forks
468
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

---

* **asyncpg version**: 0.27.0
* **PostgreSQL version**: 12
* **Python version**: 3.8
* **Platform**: Ubuntu 22
* **Do you use pgbouncer?**: no
* **Did you install asyncpg with pip?**: yes

---

It seems that asyncpg is not working correctly when using an host defined using a `bridge` network in docker. I am actually not sure the error comes from asyncpg or docker, any idea here could be really helpful.
Opening any new connection using an asyncpg connection string will result in the following error `TypeError: cannot unpack non-iterable NoneType object`, but only when using a network alias from a docker bridge network.

## More Details + how to reproduce

I have a docker-compose.yml file looking like: (simplified for the sake of this issue)
```yml
version: '3.4'

services:
postgresdb:
image: postgres:12
networks:
proxy_network:
aliases:
- postgresdb_host
env_file:
- db.env
ports:
- 3333:5432
volumes:
- ./db/postgres:/docker-entrypoint-initdb.d
- ./storage/postgres:/var/lib/postgresql/data

app_one_api:
build:
context: ${APP_ONE_DIR}
target: base
ports:
- 5000:5000
networks:
proxy_network:
aliases:
- app_one_host
restart: on-failure
command: bash -c "flask run --port 5000 --host 0.0.0.0"
volumes:
- ${APP_ONE_DIR}:/app
- ./storage/app_one/:/storage/
depends_on:
- postgresdb

app_two_api:
build:
context: ${APP_TWO_DIR}
target: prod
ports:
- 1234:8383
networks:
proxy_network:
aliases:
- app_two_host
restart: on-failure
entrypoint: bash -c "python main.py"
volumes:
- ${APP_TWO_DIR}:/app
- ./storage/app_two/:/storage/
depends_on:
- postgresdb

networks:
proxy_network:
driver: bridge
```

Where `app_one` is a Flask application using SQLAlchemy and psycopg2 (synchronous session, no asyncpg there), and `app_two` is a fastAPI application using SQLAlchemy and asyncpg.
- When I use `postgresql://postgres:postgres@postgresdb_host:5432/my_db_name` as an URI for the Flask application, everything is working well, meaning my `proxy_network` is configured correctly between `app_one` and `postgresdb` (and db is properly running)
- When I run my `app_two` outside of docker, but still run `postgresdb` container, I can properly use `postgresql+asyncpg://postgres:postgres@127.0.0.1:3333/my_db_name` and it works. (external host / ports), meaning the database container is working correctly
- When I do a HTTP call from my `app_two` container, to an endpoint of `app_one`, using `http://app_one_host:5000` as URI, it works correctly, meaning `app_two` and `app_one` are also properly linked to the bridge network.

However, using `postgresql+asyncpg://postgres:postgres@postgresdb_host:5432/my_db_name` as my database URI from within the docker container of `app_two`, here is the error I get when trying to open any new connection:

```
File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr
tr, pr = await compat.wait_for(connector, timeout=timeout)
TypeError: cannot unpack non-iterable NoneType object
```

see full error stacktrace:

```
File "/app/app/db/session.py", line 85, in check_dbs
async with self.engines[bind].begin() as connection:
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/base.py", line 66, in __aenter__
return await self.start(is_ctxmanager=True)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py", line 599, in start
await self.conn.start(is_ctxmanager=is_ctxmanager)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py", line 157, in start
await (greenlet_spawn(self.sync_engine.connect))
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 126, in greenlet_spawn
result = context.throw(*sys.exc_info())
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/future/engine.py", line 406, in connect
return super(Engine, self).connect()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3320, in connect
return self._connection_cls(self, close_with_result=close_with_result)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 96, in __init__
else engine.raw_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3399, in raw_connection
return self._wrap_pool_connect(self.pool.connect, _connection)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3366, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 327, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 894, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 493, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 273, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 388, in __init__
self.__connect()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 691, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 686, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 574, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 598, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 780, in connect
await_only(self.asyncpg.connect(*arg, **kw)),
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 68, in await_only
return current.driver.switch(awaitable)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in greenlet_spawn
value = await result
File "/usr/local/lib/python3.8/site-packages/asyncpg/connection.py", line 2092, in connect
return await connect_utils._connect(
File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 881, in _connect
return await _connect_addr(
File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 773, in _connect_addr
return await __connect_addr(params, timeout, True, *args)
File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr
tr, pr = await compat.wait_for(connector, timeout=timeout)
TypeError: cannot unpack non-iterable NoneType object
```

With all those experiments I did, I cannot find what is the issue, except that it happens only when using asyncpg + bridge network. Bridge network with `psycopg2` works fine, as well as `asyncpg` without bridge network hosts. Combining both fails.

I tried to add as much info as possible on my issue and how to reproduce, but feel free to ask anything I may have forgotten!

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par reproduire la connexion depuis app_two via le réseau bridge de docker-compose.yml et l’URI postgresdb_host. Lisez asyncpg/connect_utils.py autour de __connect_addr et comparez-le à la connexion fonctionnelle à 127.0.0.1 ; le site d’appel de l’application se trouve dans app/db/session.py. C’est terminé lorsque le chemin de connexion signalé ne lève plus le TypeError de déballage et que la cause ainsi que le comportement sont vérifiés.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
docker, docker-compose, postgresql, python
Domaine
backend, databases, networking
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
30/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.