DIRACGrid / DIRACGrid/diracx

[Feature]: Set up Alembic in diracx-db (initial structure, baseline revisions, CLI, docs)

Open
#984 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19
Forks
45
Avg merge
4d 23h
Merged PRs (30d)
12

Description

## User Story

As a DiracX administrator,
I want `python -m diracx.db upgrade` to bring my databases to the schema the running code expects,
So that upgrading never requires me to copy an `ALTER TABLE` out of a pull-request description.

And, as a DiracX contributor,
I want a how-to page telling me what to do when I add a column,
So that I can write a correct migration without reading the ADR.

As an extension developer,
I want to ship migrations for the tables my extension adds — including tables I attach to a DiracX-owned
database,
So that `diracx upgrade` on an installation with my extension migrates both DiracX's schema and mine,
and neither one tries to drop the other's tables.

## Feature Description

This is the standalone PR that introduces Alembic, agreed in discussion to land **before** #967 rather
than as part of it. #967 then becomes the first real revision, written by someone following this issue's
documentation — which is the point: it tests the docs.

Implements DX-ADR-003.

**Contributor documentation** (`docs/dev/how-to/change-a-db-schema.md`) — part of this issue, not a
follow-up, because the next person to write a migration will work from it.

### Note on extensions

Extensions register databases through the `diracx.dbs.sql` entry point, and they do so in two distinct
ways. The migration design has to handle both, and the second one is what makes this non-trivial.

**(a) A brand-new database with its own `MetaData`.** `BookkeepingDB` (lhcbdiracx),
`LollygagDB` and `MyPilotDB` (gubbins) each declare their own `DeclarativeBase`. Each gets its own
version tree, owned entirely by the extension. Nothing special is needed beyond the discovery mechanism.

**(b) Extra tables inside a DiracX-owned database.** `GubbinsJobDB(JobDB)` sets
`metadata = JobDBBase.metadata` and its `GubbinsInfo` table inherits `JobDBBase`, so `create_all`
produces DiracX's `JobDB` tables *and* `GubbinsInfo` in one pass, in one database, with foreign keys
between them.

That means the `JobDB` database's `alembic_version` has to track **two heads**: DiracX's and the
extension's. Without this, a DiracX-authored `--autogenerate` sees `GubbinsInfo` as an unknown table and
proposes `DROP TABLE GubbinsInfo`.

## Definition of Done

- [ ] `alembic` added as a dependency of `diracx-db`
- [ ] Scripts to interact with alembic
- [ ] Works with extensions
- [ ] Unit tests if possible
- [ ] A contributor unfamiliar with Alembic can follow the docs end to end without reading the ADR

## Alternatives Considered

## Related Issues

## Additional Context

Contributor guide

Open the contributing guide

Research direction

Start with DX-ADR-003 and the diracx.db entry point, then inspect the extension examples named in the issue, including JobDB and GubbinsJobDB. Use python -m diracx.db upgrade as the operational target and read docs/dev/how-to/change-a-db-schema.md as the contributor-facing deliverable. Done means the dependency, CLI scripts, extension migration handling, tests, and end-to-end documentation meet the listed definition of done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlalchemy
Domain
database, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.