sqlalchemy / sqlalchemy/alembic

git does not commit "versions" by default, so we should possibly ensure some file is present there

Open
#1,320 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

migration environment use case
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
Creating a revision fails if the versions directory is missing

Expected behavior
versions directory is created, then revision is created or a clean error message explaining the problem

To Reproduce
Please try to provide a Minimal, Complete, and Verifiable example, with the migration script and/or the SQLAlchemy tables or models involved.
See also Reporting Bugs on the website.

(main)|pim]$ pdm run alembic -c alembic.dev.ini revision -m "create product table"

Error

  Generating /projects/pim/migrations/versions/b6966ad88634_create_product_table.py ...  FAILED
Traceback (most recent call last):
  File "/projects/pim/.venv/bin/alembic", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/config.py", line 630, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/config.py", line 624, in main
    self.run_cmd(cfg, options)
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/config.py", line 601, in run_cmd
    fn(
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/command.py", line 240, in revision
    scripts = [script for script in revision_context.generate_scripts()]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/command.py", line 240, in <listcomp>
    scripts = [script for script in revision_context.generate_scripts()]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/autogenerate/api.py", line 640, in generate_scripts
    yield self._to_script(generated_revision)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/autogenerate/api.py", line 545, in _to_script
    return self.script_directory.generate_revision(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/script/base.py", line 740, in generate_revision
    self._generate_template(
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/script/base.py", line 588, in _generate_template
    util.template_to_file(src, dest, self.output_encoding, **kw)
  File "/projects/pim/.venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 39, in template_to_file
    with open(dest, "wb") as f:
         ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/projects/pim/migrations/versions/b6966ad88634_create_product_table.py'

Versions.

  • OS: MacOS 13.5.2 (22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000 arm64)
  • Python: 3.11
  • Alembic: 1.12.0
  • SQLAlchemy: 2.0.21
  • Database: Postgres
  • DBAPI: asyncpg

Additional context
This is a corner case. I initialized alembic and due to the timing, committed after the initialization but before creating the first migration. The versions directory was empty and was not preserved. I moved to a different device and ran a pull, then tried to create the initial revision. That failed.

As you may note from the above, I am using pdm and initially thought there was a pathing issue. That is not the case - I activated the virtualenv and was able to reproduce.

Obviously this is a bit of a silly issue, as it's unlikely many people are going to initialize alembic and then not create an initial revision, but it could happen to someone else. I suspect that autoinstantiating the versions directory if not present is probably the easiest behavior to include, but if that can't happen for some reason (lacking create permissions/device full/other things) catching the issue and then producing a clear error message seems like the right thing.

Have a nice day!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at alembic/script/base.py, where revision generation reaches the missing migrations/versions path, and review the revision command shown in the report. Reproduce with an initialized Alembic project whose versions directory is absent. Done means revision creation succeeds by creating the directory or reports a clear error when that is not possible, with tests covering the case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.