haskell / haskell/cabal

`cabal-validate`: Steps are not granular enough, add groups of steps

Open
#10,563 0 comments 0 reactions 0 assignees View on GitHub
cabal-validate meta: tracking re: devx type: testing
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

Each `--step` in `cabal-validate` runs multiple test suites.

This makes it difficult to pinpoint which test suite caused a failure. This also makes rerunning tests (while fixing a failure) take longer than it needs to, because you may need to wait for unrelated test suites to finish. (The `--pattern` option usually helps, though.)

**Proposal:** Make steps run a single test suite instead of multiple, and introduce 'groups' to run multiple steps.

It might be nice if groups/steps have a nesting hierarchy, e.g. `build` builds everything, `build-lib` builds `Cabal`, `Cabal-hooks`, etc., and `build-Cabal` builds `Cabal` only.

Contributor guide

Open the contributing guide

Research direction

Start with the cabal-validate entry point and trace how --step and --pattern select test suites. Map the existing multi-suite steps before deciding how individual steps and groups should be represented. Done means a single test suite can be run independently, groups can run multiple steps, and the hierarchy remains usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.