haskell / haskell/cabal

> 6 minutes is spent skipping tests in CLI (skipping individual tests takes a long time)

Open
#9,426 3 comments 2 reactions 0 assignees View on GitHub
continuous-integration type: enhancement
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

If you look at a CI log you will see for the `lib-suite` test section:

```
480 tests, 362 skipped, 0 unexpected passes, 0 unexpected fails.
```

Then if you look at a skipped test, they take > 1s to conclude to skip.

```
PackageTests\UserConfig\cabal.test.hs SKIP no cabal-install (1.24s)
```

So if 360 tests each take 1 second to skip, that is 6 minutes wasted.

Some possible improvements:

* Investigate whether running the test can be made faster
* Don't separately run `lib-suite` and `cli-suite`, so less tests are skipped each time.

Contributor guide

Open the contributing guide

Research direction

Start by examining the CI output for the lib-suite and cli-suite sections, then trace the skipped PackageTests\UserConfig\cabal.test.hs case and its reported skip timing. Compare the separate suite runs and investigate whether skipped tests can conclude faster; done means reducing the time spent skipping tests without losing coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
cli, performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.