yesodweb / yesodweb/persistent

Migration with is not idempotent when combining default=null and Maybe type

Open
#1,535 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Haskell
Stars
486
Forks
306
PR merge metrics
No merged PRs in 30d

Description

On the project that I work on, we have a default field similar to this:

SomeEntity
  name    Text Maybe    default=null

When we run migrations on the project, we always get:

ALTER TABLE "some_entity" ALTER COLUMN "name" SET DEFAULT null;

The column default gets set as 'nullable' in the database but every time we run migrations this ALTER TABLE is still output.

As with #1532 I looked at this by adding some logging here:

https://github.com/yesodweb/persistent/blob/a4b489032b9bb14a3637522aae89014b714494a4/persistent-postgresql/Database/Persist/Postgresql.hs#L1568

And looked in newcols and fst old' (the new columns vs existing columns), showing the field name and default value on the Column.

Here are the values that were logged out:

-- this is from `newcols`
(FieldNameDB {unFieldNameDB = "text_field"},Just "null")

-- this is from `fst old`
(FieldNameDB {unFieldNameDB = "text_field"},Nothing)

Different to 1532 where there is obviously a mis-match with the cases of the strings, I am less clear what is going on here. I am familiar enough with the types in persistent to know that the concept of nullable is also tracked beyond this field cDefault field here (ie on column there is cNull, there are also similar fields on other types), and I think it might be that in the case where there is Nothing for the default value the null has actually been converted into a flag to signal it being nullable somewhere else (while that hasn't happened on the parsed migration).

I'm not sure at all what the solution would be to this, but I'd be happy to also look to fix this too if I could get some steer in the right direction.

A key point to this issue is I think it is down to the fact that the field is already using Maybe in conjunction to a default null value, in this case, we may not actually need to specify default=null in our entity definitions, and that might be the solution here. But it may also be good for persistent to warn about this if you write this in your entity definitions.

I created a test to recreate this, there is an open PR with failing test here:

https://github.com/yesodweb/persistent/pull/1534

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the migration comparison logic around Database/Persist/Postgresql.hs at the linked line, then review the failing reproduction test in pull request 1534. Trace how Maybe fields, nullable state, and default values are represented in newcols and the existing columns. Done means repeated migrations no longer emit ALTER TABLE ... SET DEFAULT null for this schema, with the regression test passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.