IntersectMBO / IntersectMBO/cardano-haskell-packages
Cut down the evaluation overhead of `build-packages`
- Dominant language
- Shell
- Stars
- 36
- Forks
- 35
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 25
Description
At the moment `build-packages` builds ~6 derivations in a single Nix command. That doesn't seem so bad... except that each of them is its own independent cabal project, and so needs to do the `haskell.nix` cabal planning IFD. So in fact the evaluation time becomes significant: about 5min at the moment, and it'll be more as we add more packages to the smoke test.
(For the building we have `nixbuild.net`, which is massively parallel, but the evaluation is serial!)
It would be nice to cut this down. At the moment the only way I can think of doing that is to do a matrix build in GHA, with each package in its own job. Then the evaluation work can be done in parallel. But this would be a bit annoying, we'd need to get the list of things to build out of the flake and then call back into it again...
Contributor guide
Research direction
Start by tracing the build-packages command and the flake to see how the smoke-test derivations are enumerated, then review the GitHub Actions approach described in the issue. The change is done when package evaluations can run in parallel and the serial evaluation overhead is reduced without losing the existing smoke-test coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, haskell, shell
- Domain
- build-system, ci-cd, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100