sqlalchemy / sqlalchemy/alembic

document that URLs need to be URL escaped, and if present in alembic.ini also need percent signs doubled

Open
#700 14 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
Alembic cannot consume database URIs that contain special characters.

Expected behavior
Expected behavior is either a nice error message, or preferably consuming the database URI string without complaint.

To Reproduce

Create a new, minimal Alembic repository. In the env.py, add a line like config.set_main_option("sqlalchemy.url", "postgres://test:@#$%^&*()@localhost/test"). Then attempt to create a revision and upgrade the database.

Alternatively, I have an example repository.

Error

Traceback (most recent call last):
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/config.py", line 577, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/config.py", line 571, in main
    self.run_cmd(cfg, options)
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/config.py", line 548, in run_cmd
    fn(
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/command.py", line 298, in upgrade
    script.run_env()
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "alembic/env.py", line 12, in <module>
    config.set_main_option("sqlalchemy.url", "postgres://test:@#$%^&*()@localhost/test")
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/config.py", line 242, in set_main_option
    self.set_section_option(self.config_ini_section, name, value)
  File "/home/sky/.cache/pypoetry/virtualenvs/alembic-test-8N3_MNVT-py3.8/lib/python3.8/site-packages/alembic/config.py", line 269, in set_section_option
    self.file_config.set(section, name, value)
  File "/usr/lib/python3.8/configparser.py", line 1201, in set
    super().set(section, option, value)
  File "/usr/lib/python3.8/configparser.py", line 894, in set
    value = self._interpolation.before_set(self, section, option,
  File "/usr/lib/python3.8/configparser.py", line 402, in before_set
    raise ValueError("invalid interpolation syntax in %r at "
ValueError: invalid interpolation syntax in 'postgres://test:@#$%^&*()@localhost/test' at position 19

Versions.

  • OS: WSL 2 with Ubuntu 20.04
  • Python: 3.8.2
  • Alembic: 1.4.2
  • SQLAlchemy: 1.1.0
  • Database: Latest Postgres image on Docker
  • DBAPI:

Additional context

This is relevant because, if a DB URI's password is naively escaped
(such as with urllib.parse.quote) it will contain many many % signs. E.g. @#$%^&* will turn into %40%23%24%25%5E%26%2A. I think that ideally the config interpolation should not use % interpolation?

Assuming this is a simple issue to fix, I intend to open a PR to try and address this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating Alembic's configuration documentation and the documented use of config.set_main_option("sqlalchemy.url"). Explain URL escaping and the need to double percent signs when the URI is placed in alembic.ini, using the reproduction and urllib.parse.quote example as guidance. Done means the documented behavior and examples address both configuration paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python, sqlalchemy
Domain
databases, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.