haskell / haskell/haskell-language-server
Improving the Benchmark Suite
- Dominant language
- Haskell
- Stars
- 3k
- Forks
- 455
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 11
Description
The benchmark suite is a highly flexible performance benchmarking framework, centered
around Shake build targets. However, it has high cognitive load for HLS non-experts. In
particular, it:
* Has multiple interconnected projects: `bench`, `ghcide-bench` and `shake-bench`.
* Constructs an entire Shake build system
* Outputs large hierarchical CSV
* Has high levels of abstraction
That said, I propose the following changes:
* Consolidate `bench`, `ghcide-bench` and `shake-bench` into a single `bench` project
* Completely remove `shake`
* Replace orchestration/statistical processing with `tasty-bench`, if reasonably fits
* Use familiar benchmarking terminology:
* Examples -> Projects or Harnesses
* Experiments -> Benchmarks
* Configurations -> separate into two distinct concepts:
* Test run configuration
* HLS plugin set
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by mapping the three named projects—bench, ghcide-bench, and shake-bench—and reviewing how they construct Shake builds and produce hierarchical CSV output. Compare the current orchestration with tasty-bench, then define completion as one consolidated bench project with the proposed terminology and no Shake dependency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- build-system, performance, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100