yesodweb / yesodweb/persistent

Reducing size of generated code (compile time reduction)

Open
#1,005 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Haskell
Stars
486
Forks
306
PR merge metrics
No merged PRs in 30d

Description

After the dramatic reduction in compile times introduced by #1003, I'm thinking there may be more gains to be had.

I uploaded the template haskell output of persistent template's test file main.hs, as well as a separate file doing just a mkMigrate, here: https://gist.github.com/MaxGabriel/65027cfcea90b68b50e6ba423c619ded

share [mkPersist sqlSettings { mpsGeneric = False }, mkDeleteCascade sqlSettings { mpsGeneric = False }] [persistUpperCase|
Person json
    name Text
    age Int Maybe
    foo Foo
    address Address
    deriving Show Eq
Address json
    street Text
    city Text
    zip Int Maybe
    deriving Show Eq
NoJson
    foo Text
    deriving Show Eq
|]

share [mkPersist sqlSettings { mpsGeneric = False, mpsGenerateLenses = True }] [persistLowerCase|
Lperson json
    name Text
    age Int Maybe
    address Laddress
    deriving Show Eq
Laddress json
    street Text
    city Text
    zip Int Maybe
    deriving Show Eq
|]

migrate:

share [mkMigrate "getRequiredMigrations"] [persistLowerCase|
Mperson json
    name Text
    age Int Maybe
    address Maddress
    deriving Show Eq
Maddress json
    street Text
    city Text
    zip Int Maybe
    deriving Show Eq
|]

I'm not seeing anything jumping out as obvious as #1003 at this point. mkMigrate basically duplicates all the EntityDefs for the liftAndFixKeys trick it does (listing all entitydefs once, then listing each entity def again for each migrate function, so that could be something. (Edit: maybe a Map could be stored, and each call to migrate just references a key in the map?)

If not fundamental rethinks, there may be wins to be had by shaving off code bit by bit. For example, the new definition of fromPersistValues is much smaller than before 2.8.0. But there's still a lot of duplication with mapLeft, fieldError, the table name, and fromPersistValue being repeated each time.

fromPersistValues [x1_ausl, x2_ausm, x3_ausn, x4_auso]
        = Person
            <$>
              (Database.Persist.TH.mapLeft
                 ((Database.Persist.TH.fieldError (pack "Person")) (pack "name"))
                 . fromPersistValue)
                x1_ausl
            <*>
              (Database.Persist.TH.mapLeft
                 ((Database.Persist.TH.fieldError (pack "Person")) (pack "age"))
                 . fromPersistValue)
                x2_ausm
            <*>
              (Database.Persist.TH.mapLeft
                 ((Database.Persist.TH.fieldError (pack "Person")) (pack "foo"))
                 . fromPersistValue)
                x3_ausn
            <*>
              (Database.Persist.TH.mapLeft
                 ((Database.Persist.TH.fieldError (pack "Person")) (pack "address"))
                 . fromPersistValue)
                x4_auso

Edit: I tried removing the duplication of mapLeft fieldError and the table name, but it didn't appear to improve performance. The changes I made are here: https://github.com/yesodweb/persistent/compare/fieldErrorOptimization?expand=1 and benchmarks here: https://gist.github.com/MaxGabriel/5badd57a3ded7bac2ba43cc3c2a2a3b3

I don't have a great idea of what reduces compile times besides reducing raw LOC/expressions to parse, which isn't ideal. I'm not sure if the benchmarks will reliably show micro-improvements, but we'll see. I will probably work on this over time, but I think opening the issue is worthwhile so people can have an awareness of just how much code is generated, what it looks like, and realize there is potential for improving it.

  • Your OS name and version - MacOS Mojave
  • The versions of tools you're using (e.g. stack, ghc). - stack 1.9.3, lts-14.1
  • The versions of dependencies you're using - Persistent 2.8.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the uploaded Template Haskell output for main.hs and the separate mkMigrate output, then inspect how mkPersist and mkMigrate generate EntityDefs and fromPersistValues. The issue has no agreed implementation or named source files; done would require a concrete, benchmarked reduction in generated code or compile time.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
build-system, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.