tortoise / tortoise/tortoise-orm
Postgres connection string (dsn) with multi-host does not work
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Describe the bug
Postgres connection string (dsn) with multi-host does not work.
Example dsn:
postgres://fake-user:fake-password@127.1.2.3,127.1.2.4:12345/fake-db?target_session_attrs=read-write
To Reproduce
Run something like this:
import asyncio
from tortoise import Tortoise, transactions
async def main():
config = {
"connections": {
# !!!
# Using a DB_URL string with 2 hosts (primary + replica)
"default": "postgres://postgres:qwerty123@host1.com,host2.com:5432/test",
},
"apps": {
"my_app": {
"models": ["__main__"],
# If no default_connection specified, defaults to 'default'
"default_connection": "default",
}
},
"routers": ["path.router1", "path.router2"],
"use_tz": False,
"timezone": "UTC",
}
await Tortoise.init(config=config)
async with transactions.in_transaction() as connection:
await connection.execute_query("SELECT 1;")
if __name__ == '__main__':
asyncio.run(main())
Traceback (most recent call last):
File "contrib/python/starlette/starlette/routing.py", line 732, in lifespan
async with self.lifespan_context(app) as maybe_state:
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/fastapi/fastapi/routing.py", line 131, in merged_lifespan
async with original_context(app) as maybe_original_state:
^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "src/api/app.py", line 30, in lifespan_v2
await wait_until_connected()
File "src/core/database.py", line 73, in wait_until_connected
async with transactions.in_transaction() as connection:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/tortoise-orm/tortoise/backends/base/client.py", line 273, in __aenter__
await self.ensure_connection()
File "contrib/python/tortoise-orm/tortoise/backends/base/client.py", line 270, in ensure_connection
await self.connection._parent.create_connection(with_db=True)
File "contrib/python/tortoise-orm/tortoise/backends/asyncpg/client.py", line 59, in create_connection
self._pool = await self.create_pool(password=self.password, **self._template)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/tortoise-orm/tortoise/backends/asyncpg/client.py", line 70, in create_pool
return await asyncpg.create_pool(None, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/pool.py", line 418, in _async__init__
await self._initialize()
File "contrib/python/asyncpg/asyncpg/pool.py", line 445, in _initialize
await first_ch.connect()
File "contrib/python/asyncpg/asyncpg/pool.py", line 132, in connect
self._con = await self._pool._get_new_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/pool.py", line 517, in _get_new_connection
con = await self._connect(
^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/connection.py", line 2421, in connect
return await connect_utils._connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/connect_utils.py", line 1075, in _connect
raise last_error or exceptions.TargetServerAttributeNotMatched(
File "contrib/python/asyncpg/asyncpg/connect_utils.py", line 1049, in _connect
conn = await _connect_addr(
^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/connect_utils.py", line 886, in _connect_addr
return await __connect_addr(params, True, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/connect_utils.py", line 931, in __connect_addr
tr, pr = await connector
^^^^^^^^^^^^^^^
File "contrib/python/asyncpg/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
tr, pr = await loop.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/asyncio/base_events.py", line 1083, in create_connection
infos = await self._ensure_resolved(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/asyncio/base_events.py", line 1466, in _ensure_resolved
return await loop.getaddrinfo(host, port, family=family, type=type,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/asyncio/base_events.py", line 905, in getaddrinfo
return await self.run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "contrib/tools/python3/Lib/socket.py", line 978, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
Expected behavior
It should work, like it works in asyncpg if we pass dsn url directly there.
Additional context
The bug is actually here during parsing:
https://github.com/tortoise/tortoise-orm/blob/develop/tortoise/backends/base/config_generator.py#L164
How it should be for Postgres and asyncpg:
https://github.com/MagicStack/asyncpg/blob/master/asyncpg/connect_utils.py#L272
I'd suggest to use this util inside if asyncpg engine is selected in the config.
Stack:
- tortoise-orm == 0.23.0
- asyncpg == 0.30.0
- Postgres
Possible fix
I've made a small workaround in our project just for this case:
result = tortoise.backends.base.config_generator.expand_db_url(db_url=db_url)
result["credentials"]["host"] = result["credentials"]["host"].split(",")
config["connections"]["default"] = result
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at tortoise/backends/base/config_generator.py around line 164 and compare its PostgreSQL URL parsing with asyncpg/connect_utils.py around line 272. Reproduce the multi-host DSN case from the report and verify that the resulting configuration preserves both hosts so the connection behaves like asyncpg when given the DSN directly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100