haskell / haskell/cabal

"cabal test without-tests with-tests" won't run tests

Open
#11,858 5 comments 1 reaction 1 assignee Claimed by @zlonast View on GitHub
type: bug type: discussion
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

## Expected

Given:
1. A project with two packages,
2. One package has tests,
3. The other package does not have tests,

When I run `cabal test pkg-with-tests pkg-without-tests`,

Then I expect tests to run for the package with tests.

## Actual

No tests run.

```
$ cabal test pkg-with-tests pkg-without-tests
No tests to run for the package pkg-without-tests-0.1.0.0
```

And perhaps even worse, cabal exits successfully.

```
$ cabal test pkg-with-tests pkg-without-tests
No tests to run for the package pkg-without-tests-0.1.0.0

$ echo $?
0
```

### Compare to:

```
$ cabal test all
Build profile: -w ghc-9.10.3 -O1
In order, the following will be built (use -v for more details):
- pkg-with-tests-0.1.0.0 (test:pkg-with-tests-test) (first run)
Preprocessing test suite 'pkg-with-tests-test' for pkg-with-tests-0.1.0.0...
Building test suite 'pkg-with-tests-test' for pkg-with-tests-0.1.0.0...
Running 1 test suites...
Test suite pkg-with-tests-test: RUNNING...
Test suite pkg-with-tests-test: PASS
Test suite logged to:
/home/b/tmp/cabal-two-pkg-repro/dist-newstyle/build/x86_64-linux/ghc-9.10.3/pkg-with-tests-0.1.0.0/t/pkg-with-tests-test/test/pkg-with-tests-0.1.0.0-pkg-with-tests-test.log
1 of 1 test suites (1 of 1 test cases) passed.
```

### Discussion

It makes sense to print an info statement, but I think it's unnecessary to cancel all the other tests if one package doesn't have any. And exiting successfully is a foot-gun.

## To reproduce

Download and unpack the repro, run the commands given above.

[repro.tar.gz](https://github.com/user-attachments/files/28090183/repro.tar.gz)

## System info
```
$ cabal --version
cabal-install version 3.16.0.0
compiled using version 3.16.0.0 of the Cabal library (in-tree)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.