IntersectMBO / IntersectMBO/cardano-haskell-packages

Cut down the evaluation overhead of `build-packages`

Open
#171 2 comments 0 reactions 0 assignees View on GitHub
CI
Dominant language
Shell
Stars
36
Forks
35
Avg merge
1d 4h
Merged PRs (30d)
25

Description

At the moment `build-packages` builds ~6 derivations in a single Nix command. That doesn't seem so bad... except that each of them is its own independent cabal project, and so needs to do the `haskell.nix` cabal planning IFD. So in fact the evaluation time becomes significant: about 5min at the moment, and it'll be more as we add more packages to the smoke test.

(For the building we have `nixbuild.net`, which is massively parallel, but the evaluation is serial!)

It would be nice to cut this down. At the moment the only way I can think of doing that is to do a matrix build in GHA, with each package in its own job. Then the evaluation work can be done in parallel. But this would be a bit annoying, we'd need to get the list of things to build out of the flake and then call back into it again...

Contributor guide

Open the contributing guide

Research direction

Start by tracing the build-packages command and the flake to see how the smoke-test derivations are enumerated, then review the GitHub Actions approach described in the issue. The change is done when package evaluations can run in parallel and the serial evaluation overhead is reduced without losing the existing smoke-test coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, haskell, shell
Domain
build-system, ci-cd, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.