yesodweb / yesodweb/persistent

Unsigned integer support

Open
#831 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug MySQL
Dominant language
Haskell
Stars
486
Forks
306
PR merge metrics
No merged PRs in 30d

Description

I'll demonstrate this bug with an Esqueleto query but the root cause lies in the Persistent data model.

Symptoms

My (existing) MariaDB database schema has a nullable unsigned int column, which I model in Persistent as Word32 Maybe. When I select $ from $ \table -> pure $ coalesceDefault [table ^. TableColumn] (val 0) with Esqueleto, the program crashes at runtime with PersistMarshalError "Failed to parse Haskell type 'Word32'; expected integer from database, but received: PersistDouble 0.0. Potential solution: Check that your database schema matches your Persistent model definitions."

This happens with both persistent-mysql and persistent-mysql-haskell backends.

Cause

The generated SQL query is SELECT COALESCE(table_column, ?) FROM table with parameters [0 :: Int64]. Despite Esqueleto's type system ensuring that both arguments to COALESCE have the same type, the underlying PersistField instance for Word32 changes the type of the literal 0 to Int64, while the table column stays unsigned. MariaDB decides that the type of COALESCE(unsigned_int, signed_int) is decimal(10,0), which Persistent decodes as PersistDouble. This triggers an error in the PersistField instance for Word32.

Possible solutions
  1. Decode decimal(_,0) as an integer. This is somewhat questionable because the value could exceed the range of the target type, but in such cases, we could always throw an error or return a PersistRational. Or perhaps decimal(m,n) should always be decoded as PersistRational and integral types should accept PersistRationals that represent integers.

  2. Extend the PersistField Word32 instance to accept (and truncate) PersistDouble values, the same way as the Int instance does. This has all the problems of the above solution, plus it's ugly and will silently lose precision in 64-bit values, given the 53-bit mantissa of Double. (Although this is probably more problematic for Int than Word32.)

  3. Extend PersistValue with a new constructor PersistWord64 Word64 and SqlType with SqlWord32 and SqlWord64 and then change instance PersistField Word32 to use the appropriate representation. This is the cleanest option, IMO, but it requires adjustments to all backends.

Discussion

You could argue that I'm using a database schema that does not correspond to the Persistent model -- the PersistField Word32 instance assumes that the underlying column type is a signed 64-bit bigint. However, 1) you don't always control your underlying database 2) if it says Word, it should really be unsigned and work correctly.

So this could also be seen as a complaint that the Word32 instance represents values neither as a words, nor in 32 bits.

I'm not listing any software versions; I believe this is universally applicable to any MySQL/MariaDB deployment.

I'd submit a PR but I don't want to put in the work without knowing your opinion. Does a patch adding explicit unsigned int support have a chance to be accepted?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue identifies Persistent's PersistField Word32, PersistValue, and SqlType, along with the persistent-mysql and persistent-mysql-haskell backends, as the relevant entry points. First compare the three proposed representations against MariaDB's decimal handling; done requires a decided unsigned-integer behavior that works across the affected backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell, mariadb, mysql
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.