tortoise / tortoise/tortoise-orm

Asyncpg does not support sslmode as a query parameter in a database URL

Open
#1,376 2 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Describe the bug
Given a Postgres database URL, such as:

postgres://user:password@db/mydatabase?sslmode=disable

The following error(s) occur when trying to establish a connection to the database via aerich upgrade:

Traceback (most recent call last):
  File "/usr/local/bin/aerich", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aerich/cli.py", line 258, in main
    cli()
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1654, in invoke
    super().invoke(ctx)
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aerich/cli.py", line 31, in wrapper
    loop.run_until_complete(f(*args, **kwargs))
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aerich/cli.py", line 77, in cli
    await command.init()
  File "/usr/local/lib/python3.11/site-packages/aerich/__init__.py", line 37, in init
    await Migrate.init(self.tortoise_config, self.app, self.location)
  File "/usr/local/lib/python3.11/site-packages/aerich/migrate.py", line 83, in init
    last_version = await cls.get_last_version()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aerich/migrate.py", line 64, in get_last_version
    return await Aerich.filter(app=cls.app).first()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/queryset.py", line 1008, in _execute
    instance_list = await self._db.executor_class(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/base/executor.py", line 131, in execute_select
    _, raw_results = await self.db.execute_query(query.get_sql())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/base_postgres/client.py", line 34, in _translate_exceptions
    return await self._translate_exceptions(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/asyncpg/client.py", line 82, in _translate_exceptions
    return await func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/asyncpg/client.py", line 129, in execute_query
    async with self.acquire_connection() as connection:
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/base/client.py", line 326, in __aenter__
    await self.ensure_connection()
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/base/client.py", line 322, in ensure_connection
    await self.client.create_connection(with_db=True)
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/asyncpg/client.py", line 59, in create_connection
    self._pool = await self.create_pool(password=self.password, **self._template)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tortoise/backends/asyncpg/client.py", line 65, in create_pool
    return await asyncpg.create_pool(None, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/pool.py", line 409, in _async__init__
    await self._initialize()
  File "/usr/local/lib/python3.11/site-packages/asyncpg/pool.py", line 437, in _initialize
    await first_ch.connect()
  File "/usr/local/lib/python3.11/site-packages/asyncpg/pool.py", line 129, in connect
    self._con = await self._pool._get_new_connection()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/pool.py", line 507, in _get_new_connection
    con = await connection.connect(
                ^^^^^^^^^^^^^^^^^^^
TypeError: connect() got an unexpected keyword argument 'sslmode'

To Reproduce

  1. See description for an example DATABASE_URL to use
  2. Use the following TORTOISE_ORM config:
TORTOISE_ORM = {
    "connections": {
        "default": database_url
    },
    "apps": {
        "models": {
            "models": ["your_app.models", "aerich.models"],
            "default_connection": "default",
        },
    },
}
  1. Attempt to run migrations, or execute a query against the database
  2. Error should occur

Expected behavior
I expect the database connection to establish successfully using the sslmode provided in the DSN.

Additional context
I ran into this bug while trying to deploy to Fly.io, which automatically appends ?sslmode=disable to all their database URLs for their Postgres service. It also seems like this was an issue in SQLalchemy. Here are a few GitHub issues of note:

https://github.com/MagicStack/asyncpg/issues/737
https://github.com/sqlalchemy/sqlalchemy/issues/6275

I corrected the behavior by manually parsing the database URL and providing a full configuration object:

TORTOISE_ORM = {
    "connections": {
        "default": {
            "engine": "tortoise.backends.asyncpg",
            "credentials": {
                **database_info,  # database-info is a dict generated via a regex to extract the database_url components
                "port": database_info.get("port") or "5432",
                "ssl": False,  # this is the thing that should have been set due to sslmode=disable
            },
        }
    },
    "apps": {
        "models": {
            "models": ["your_app.models", "aerich.models"],
            "default_connection": "default",
        },
    },
}

...but obviously this is not ideal. Any help would be appreciated!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the asyncpg backend entry points named in the traceback, especially tortoise/backends/asyncpg/client.py and its create_connection/create_pool flow. Reproduce the DATABASE_URL containing ?sslmode=disable, then verify that migrations or a query establish the PostgreSQL connection without passing sslmode as an unsupported argument.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.