IQSS / IQSS/dataverse

Flyway: disable DDL generation from EclipseLink

Open
#5,871 14 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Component: Code Infrastructure Feature: None Size: 80 Type: Suggestion
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

This is a successor issue to #5862.
This kind of relates to #5854.
This should be read by @pdurbin, @MrK191, @donsizemore and @djbrooke at least.

Since the introduction of Flyway in #5344, only minor modifications happened to the schema.
In #5822 a new column has been added and since it had been merged, the develop branch no longer deploys on new "installations".

Some background on the cause:
EclipseLink ORM generates a DDL script from all models on startup and (due to this setting in persistance.xml) creates new tables when not existing. It does not update existing tables. This job is left to Flyway, which is one reason why it had been introduced.

Flyway throws exceptions due to a conflict, trying to change something already present. In #5822, a new column should be added, but on new deployments, EclipseLink already generated the mapping. Thus the deployment fails.

How to get along with this:
Obviously, one could just use workarounds as #5866 does. It would be better to have a discussion and decision on how to deal with database model, mappings, EclipseLink and Flyway in the future, write down a process and adapt.

IMHO there is only one "good" solution, but this is only a proposal and needs to be discussed and/or modified:

  1. Do not let EclipseLink generate DDL anymore.
  2. Get a base model mapping in the codebase as a proper baseline. This, ideally, should be the exact mapping as it has been when Flyway had been merged.
  3. Continue to write migrations in SQL or Java with Flyway, as already done. Document this part of the process if not already done sufficiently.
  4. To ensure 100% compatibility between tables and models, the mapping - now maintained with Flyway as migration files - needs to be validated regularly. This is an ideal task for CI on jenkins.dataverse.org.

It might be an option to switch from EclipseLink JPA to Hibernate JPA as it offers easier to use validation mechanisms than EclipseLink does.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading src/main/resources/META-INF/persistence.xml, the Flyway introduction in #5344, and the migration linked from #5822 to understand how EclipseLink DDL generation conflicts with migrations. Review the discussion and related issues before proposing a decision; done would require an agreed database-mapping process, documented responsibilities, and any resulting implementation or validation work.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.