lspitzner / lspitzner/brittany
Improve performance - low hanging stuff?
Nobody has claimed this yet.
- Dominant language
- Haskell
- Stars
- 686
- Forks
- 69
- PR merge metrics
- No merged PRs in 30d
Description
Even with the fix for the quadratic performance issue, brittany easily takes in the magnitude of seconds on large (1k+ loc) inputs. I think there is room for improvement.
Testing on a 1800loc module, I see the following stats:
+RTS -s output:
3,307,514,304 bytes allocated in the heap
1,105,201,976 bytes copied during GC
124,529,352 bytes maximum residency (18 sample(s))
513,336 bytes maximum slop
295 MB total memory in use (0 MB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 3041 colls, 0 par 1.480s 1.479s 0.0005s 0.1567s
Gen 1 18 colls, 0 par 0.003s 0.008s 0.0005s 0.0015s
INIT time 0.000s ( 0.000s elapsed)
MUT time 3.097s ( 3.268s elapsed)
GC time 1.483s ( 1.488s elapsed)
RP time 0.000s ( 0.000s elapsed)
PROF time 0.000s ( 0.000s elapsed)
EXIT time 0.003s ( 0.013s elapsed)
Total time 4.583s ( 4.769s elapsed)
%GC time 32.4% (31.2% elapsed)
Alloc rate 1,068,089,845 bytes per MUT second
Productivity 67.6% of total user, 68.8% of total elapsed
Max residency is a bit high, but productivity still looks fine. Looking at the prof output:
total time = 3.17 secs (3165 ticks @ 1000 us, 1 processor)
total alloc = 1,929,655,872 bytes (excludes profiling overheads)
COST CENTRE MODULE SRC %time %alloc
filterAnns Language.Haskell.Brittany.Internal.LayouterBasics src/Language/Haskell/Brittany/Internal/LayouterBasics.hs:(229,1)-(230,67) 23.3 0.3
everything Data.Generics.Schemes src/Data/Generics/Schemes.hs:104:1-59 16.0 24.1
censor Control.Monad.Writer.Class Control/Monad/Writer/Class.hs:(99,1)-(101,17) 13.9 19.9
runMemoStateT Control.Monad.Trans.Memo.State Control/Monad/Trans/Memo/State.hs:(56,1)-(58,23) 9.9 12.4
iterTM Control.Monad.Trans.Free src/Control/Monad/Trans/Free.hs:(317,1)-(321,21) 3.3 5.3
<&> Language.Haskell.Brittany.Internal.PreludeUtils src/Language/Haskell/Brittany/Internal/PreludeUtils.hs:61:1-17 3.2 2.6
parseModuleApiAnnsWithCppInternal Language.Haskell.GHC.ExactPrint.Parsers src/Language/Haskell/GHC/ExactPrint/Parsers.hs:(232,1)-(252,58) 3.0 4.1
…
filterAnns still is high up - probably via hasAnyCommentsBelow. I think that part can definitely be improved. For a different input (with 1000 one-line type-sig + one-line function) it looks like:
total time = 1.49 secs (1493 ticks @ 1000 us, 1 processor)
total alloc = 999,113,800 bytes (excludes profiling overheads)
COST CENTRE MODULE SRC %time %alloc
everything Data.Generics.Schemes src/Data/Generics/Schemes.hs:104:1-59 18.6 21.9
runMemoStateT Control.Monad.Trans.Memo.State Control/Monad/Trans/Memo/State.hs:(56,1)-(58,23) 15.3 13.5
censor Control.Monad.Writer.Class Control/Monad/Writer/Class.hs:(99,1)-(101,17) 14.7 16.8
layoutBriDocM Language.Haskell.Brittany.Internal.Backend src/Language/Haskell/Brittany/Internal/Backend.hs:(75,1)-(258,20) 3.9 2.8
parseModuleApiAnnsWithCppInternal Language.Haskell.GHC.ExactPrint.Parsers src/Language/Haskell/GHC/ExactPrint/Parsers.hs:(232,1)-(252,58) 3.1 4.1
…
We use monad-memo in the most basic way, but unfortunately the interface of that package does not allow us to switch to a more efficient backend (e.g. vector-based) easily, afaict. And if we start refactoring, we may as well run our own memoization and remove the monad-memo dep (see #2).
censor is called via ghc-exactprint, so I suspect out of scope. Still might be worth investigating that one at some point.
Also, parallelization - the top-level module elements can be processed independently. I don't like fighting for constant factors before I have to, but then it could be relatively easy here.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported large-input benchmarks and profiling output. Inspect filterAnns and hasAnyCommentsBelow in src/Language/Haskell/Brittany/Internal/LayouterBasics.hs, then review monad-memo usage and the references to #2; compare alternatives against the listed cost centres. Done means a measured performance improvement with profiling evidence and no formatting regressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- performance, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100