yesodweb / yesodweb/persistent
Unsigned integer support
Nobody has claimed this yet.
- Dominant language
- Haskell
- Stars
- 486
- Forks
- 306
- PR merge metrics
- No merged PRs in 30d
Description
I'll demonstrate this bug with an Esqueleto query but the root cause lies in the Persistent data model.
Symptoms
My (existing) MariaDB database schema has a nullable unsigned int column, which I model in Persistent as Word32 Maybe. When I select $ from $ \table -> pure $ coalesceDefault [table ^. TableColumn] (val 0) with Esqueleto, the program crashes at runtime with PersistMarshalError "Failed to parse Haskell type 'Word32'; expected integer from database, but received: PersistDouble 0.0. Potential solution: Check that your database schema matches your Persistent model definitions."
This happens with both persistent-mysql and persistent-mysql-haskell backends.
Cause
The generated SQL query is SELECT COALESCE(table_column, ?) FROM table with parameters [0 :: Int64]. Despite Esqueleto's type system ensuring that both arguments to COALESCE have the same type, the underlying PersistField instance for Word32 changes the type of the literal 0 to Int64, while the table column stays unsigned. MariaDB decides that the type of COALESCE(unsigned_int, signed_int) is decimal(10,0), which Persistent decodes as PersistDouble. This triggers an error in the PersistField instance for Word32.
Possible solutions
-
Decode
decimal(_,0)as an integer. This is somewhat questionable because the value could exceed the range of the target type, but in such cases, we could always throw an error or return aPersistRational. Or perhapsdecimal(m,n)should always be decoded asPersistRationaland integral types should acceptPersistRationals that represent integers. -
Extend the
PersistField Word32instance to accept (and truncate)PersistDoublevalues, the same way as theIntinstance does. This has all the problems of the above solution, plus it's ugly and will silently lose precision in 64-bit values, given the 53-bit mantissa ofDouble. (Although this is probably more problematic forIntthanWord32.) -
Extend
PersistValuewith a new constructorPersistWord64 Word64andSqlTypewithSqlWord32andSqlWord64and then changeinstance PersistField Word32to use the appropriate representation. This is the cleanest option, IMO, but it requires adjustments to all backends.
Discussion
You could argue that I'm using a database schema that does not correspond to the Persistent model -- the PersistField Word32 instance assumes that the underlying column type is a signed 64-bit bigint. However, 1) you don't always control your underlying database 2) if it says Word, it should really be unsigned and work correctly.
So this could also be seen as a complaint that the Word32 instance represents values neither as a words, nor in 32 bits.
I'm not listing any software versions; I believe this is universally applicable to any MySQL/MariaDB deployment.
I'd submit a PR but I don't want to put in the work without knowing your opinion. Does a patch adding explicit unsigned int support have a chance to be accepted?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue identifies Persistent's PersistField Word32, PersistValue, and SqlType, along with the persistent-mysql and persistent-mysql-haskell backends, as the relevant entry points. First compare the three proposed representations against MariaDB's decimal handling; done requires a decided unsigned-integer behavior that works across the affected backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell, mariadb, mysql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100