lspitzner / lspitzner/brittany

Improve performance - low hanging stuff?

Open
#57 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Haskell
Stars
686
Forks
69
PR merge metrics
No merged PRs in 30d

Description

Even with the fix for the quadratic performance issue, brittany easily takes in the magnitude of seconds on large (1k+ loc) inputs. I think there is room for improvement.

Testing on a 1800loc module, I see the following stats:

+RTS -s output:

   3,307,514,304 bytes allocated in the heap
   1,105,201,976 bytes copied during GC
     124,529,352 bytes maximum residency (18 sample(s))
         513,336 bytes maximum slop
             295 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0      3041 colls,     0 par    1.480s   1.479s     0.0005s    0.1567s
  Gen  1        18 colls,     0 par    0.003s   0.008s     0.0005s    0.0015s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    3.097s  (  3.268s elapsed)
  GC      time    1.483s  (  1.488s elapsed)
  RP      time    0.000s  (  0.000s elapsed)
  PROF    time    0.000s  (  0.000s elapsed)
  EXIT    time    0.003s  (  0.013s elapsed)
  Total   time    4.583s  (  4.769s elapsed)

  %GC     time      32.4%  (31.2% elapsed)

  Alloc rate    1,068,089,845 bytes per MUT second

  Productivity  67.6% of total user, 68.8% of total elapsed

Max residency is a bit high, but productivity still looks fine. Looking at the prof output:


	total time  =        3.17 secs   (3165 ticks @ 1000 us, 1 processor)
	total alloc = 1,929,655,872 bytes  (excludes profiling overheads)

COST CENTRE                       MODULE                                             SRC                                                                         %time %alloc

filterAnns                        Language.Haskell.Brittany.Internal.LayouterBasics  src/Language/Haskell/Brittany/Internal/LayouterBasics.hs:(229,1)-(230,67)    23.3    0.3
everything                        Data.Generics.Schemes                              src/Data/Generics/Schemes.hs:104:1-59                                        16.0   24.1
censor                            Control.Monad.Writer.Class                         Control/Monad/Writer/Class.hs:(99,1)-(101,17)                                13.9   19.9
runMemoStateT                     Control.Monad.Trans.Memo.State                     Control/Monad/Trans/Memo/State.hs:(56,1)-(58,23)                              9.9   12.4
iterTM                            Control.Monad.Trans.Free                           src/Control/Monad/Trans/Free.hs:(317,1)-(321,21)                              3.3    5.3
<&>                               Language.Haskell.Brittany.Internal.PreludeUtils    src/Language/Haskell/Brittany/Internal/PreludeUtils.hs:61:1-17                3.2    2.6
parseModuleApiAnnsWithCppInternal Language.Haskell.GHC.ExactPrint.Parsers            src/Language/Haskell/GHC/ExactPrint/Parsers.hs:(232,1)-(252,58)               3.0    4.1
…

filterAnns still is high up - probably via hasAnyCommentsBelow. I think that part can definitely be improved. For a different input (with 1000 one-line type-sig + one-line function) it looks like:

	total time  =        1.49 secs   (1493 ticks @ 1000 us, 1 processor)
	total alloc = 999,113,800 bytes  (excludes profiling overheads)

COST CENTRE                       MODULE                                             SRC                                                                         %time %alloc

everything                        Data.Generics.Schemes                              src/Data/Generics/Schemes.hs:104:1-59                                        18.6   21.9
runMemoStateT                     Control.Monad.Trans.Memo.State                     Control/Monad/Trans/Memo/State.hs:(56,1)-(58,23)                             15.3   13.5
censor                            Control.Monad.Writer.Class                         Control/Monad/Writer/Class.hs:(99,1)-(101,17)                                14.7   16.8
layoutBriDocM                     Language.Haskell.Brittany.Internal.Backend         src/Language/Haskell/Brittany/Internal/Backend.hs:(75,1)-(258,20)             3.9    2.8
parseModuleApiAnnsWithCppInternal Language.Haskell.GHC.ExactPrint.Parsers            src/Language/Haskell/GHC/ExactPrint/Parsers.hs:(232,1)-(252,58)               3.1    4.1
…

We use monad-memo in the most basic way, but unfortunately the interface of that package does not allow us to switch to a more efficient backend (e.g. vector-based) easily, afaict. And if we start refactoring, we may as well run our own memoization and remove the monad-memo dep (see #2).

censor is called via ghc-exactprint, so I suspect out of scope. Still might be worth investigating that one at some point.

Also, parallelization - the top-level module elements can be processed independently. I don't like fighting for constant factors before I have to, but then it could be relatively easy here.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported large-input benchmarks and profiling output. Inspect filterAnns and hasAnyCommentsBelow in src/Language/Haskell/Brittany/Internal/LayouterBasics.hs, then review monad-memo usage and the references to #2; compare alternatives against the listed cost centres. Done means a measured performance improvement with profiling evidence and no formatting regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
performance, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.