agrafix / agrafix/superrecord

Compilation performance degrades rapidly while increasing record length

Aperta
#12 6 commenti 2 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Haskell
Stelle
83
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Compilation performance is evident on records of bigger size, just one record of 30-50 fields is able to make GHC swap. There's a branch and a script I've used to dig more info on this:
https://github.com/dredozubov/superrecord/tree/ghc-test-case
https://github.com/dredozubov/superrecord/blob/ghc-test-case/build-all.sh

I did a small investigation on this. Tried it with GHC 8.0.2 and 8.2.1(it can be done with `allow-newer`). Building it with GHC HEAD is not currently possible due to the broken dependencies. Here we go:
https://github.com/dredozubov/superrecord/blob/ghc-test-case/test/Spec.hs.in#L451 construction of a record with 35 fields take up to 2 minutes

```
!!! Chasing dependencies: finished in 11.20 milliseconds, allocated 15.786 megabytes
!!! Parser [Spec]: finished in 1.62 milliseconds, allocated 2.921 megabytes
!!! Renamer/typechecker [Spec]: finished in 5053.19 milliseconds, allocated 4453.209 megabytes
!!! Desugar [Spec]: finished in 8061.64 milliseconds, allocated 12993.168 megabytes
!!! Simplifier [Spec]: finished in 9113.55 milliseconds, allocated 10165.327 megabytes
!!! Specialise [Spec]: finished in 12565.69 milliseconds, allocated 8482.198 megabytes
OverSatApps = False}) [Spec]: finished in 12262.97 milliseconds, allocated 13919.365 megabytes
!!! Simplifier [Spec]: finished in 42090.28 milliseconds, allocated 42521.308 megabytes
!!! Simplifier [Spec]: finished in 17468.49 milliseconds, allocated 16998.313 megabytes
!!! Simplifier [Spec]: finished in 21349.30 milliseconds, allocated 32972.807 megabytes
!!! Float inwards [Spec]: finished in 804.30 milliseconds, allocated 1908.825 megabytes
!!! Called arity analysis [Spec]: finished in 1986.35 milliseconds, allocated 1895.883 megabytes
!!! Simplifier [Spec]: finished in 1644.02 milliseconds, allocated 2605.471 megabytes
!!! Demand analysis [Spec]: finished in 870.99 milliseconds, allocated 1954.719 megabytes
!!! Worker Wrapper binds [Spec]: finished in 1924.12 milliseconds, allocated 2049.162 megabytes
!!! Simplifier [Spec]: finished in 3616.14 milliseconds, allocated 3650.656 megabytes
OverSatApps = True}) [Spec]: finished in 1670.04 milliseconds, allocated 3925.652 megabytes
!!! Common sub-expression [Spec]: finished in 857.66 milliseconds, allocated 1020.030 megabytes
!!! Float inwards [Spec]: finished in 634.74 milliseconds, allocated 1013.631 megabytes
!!! Simplifier [Spec]: finished in 2484.37 milliseconds, allocated 2723.176 megabytes
!!! CoreTidy [Spec]: finished in 254.93 milliseconds, allocated 405.708 megabytes
!!! CorePrep [Spec]: finished in 163.65 milliseconds, allocated 463.810 megabytes
!!! CodeGen [Spec]: finished in 425.96 milliseconds, allocated 474.554 megabytes
```

`RecCopy` produces a huge amount of coercions: https://gist.github.com/dredozubov/6ce629d1ec16ac32e9a987ffefda2a2a
There's a lot of rewriting going on(a lot of heavy inlining in the library). Here are some core2core dumps with their respective size:

```
% ls -ls dump-35/test/Spec.verbose-core2core.split/ | tail -14 | awk '{print $1, $10}'
8 S-00
13264 S.01-simplifier
49272 S.02-levels-added
5976 S.03-float-out
42720 S.04-simplifier
41784 S.05-simplifier
8416 S.06-simplifier
8416 S.07-float-inwards
8416 S.08-simplifier
8472 S.09-simplifier
12072 S.10-levels-added
1936 S.11-float-out
1944 S.12-float-inwards
1832 S.13-simplifier
```

These dumps are too big to add as gists, so I've uploaded this tar archive instead: https://www.dropbox.com/s/pql4qgm5lplbe38/dump-35.tar.gz?dl=0

It's possible to rebuild all of this with `nix-shell -p python3 --run "ghc=/Users/dr/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc m=25 n=35 ./build-all.sh -package-db ~/.stack/snapshots/x86_64-osx/lts-8.20/8.0.2/pkgdb/"`, skipping the nix-shell bit if you have python3 installed on your system and substituting the `ghc` variable and `package-db` with correct values for you system. `m` and `n` variables mean it'll rebuild the module 10 with records of length 25 to 35.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.