haskell / haskell/cabal

`cabal test` doesn't always solve for tests (unless `--enable-tests` or `tests: True` is given)

Open
#7,883 27 comments 5 reactions 0 assignees View on GitHub
attention: pr-welcome cabal-install: cmd/test cabal-install: solver type: enhancement
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

@ulysses4ever's exec summary:

- this issue (and its variations) has been discussed on multiple occasions (#5079, #3923)

- a desirable converging point in the design space seems to be, as of now ([as put by grayjay](https://github.com/haskell/cabal/issues/5079#issuecomment-1170994829)):

> By default, `cabal build` would solve for tests but not build them, and `cabal test` would solve for tests and build them

- the main counterpoint to the idea above ([as put by fgaz](https://github.com/haskell/cabal/issues/5079#issuecomment-1159507404)):

> It could happen that there are no build plans with tests enabled but there are with tests disabled

- it feels like we get a better UX if we follow the idea still, and the answer to the counterpoint ([as put by Mikolaj](https://github.com/haskell/cabal/issues/5079#issuecomment-1171119618)):

> [In the case of no plan including tests or benchmarks exists,] the user can manually disable tests or benchmarks

- anecdotally, many power users already put `tests: True` in their global config and deal with the above mentioned problem on the case by case basis; we just enshrine this practice in the default;

- our best hope for implementing something like that is, currently, #7829 but it looks like [it needs a new champion](https://github.com/haskell/cabal/pull/7829#issuecomment-1564299115)...

---

**Describe the bug**

I'm dog-fooding current dev cabal from master branch.

Solver is not cooperating. Cabal says "solver picked a plan that does not include the test suites" despite there being such a plan, which can be seen when forcing with 'tests: True'.

**To Reproduce**

Clone

https://github.com/Mikolaj/mostly-harmless/commit/3feb586519b6fbe87aa1867700206b1af0bd600f

Run the following (it seems, a fresh package store is not required after all)

cabal test -w ghc-9.2.1

It shows

```
Resolving dependencies...
Error: cabal: Cannot test the package mostly-harmless-0.1.0.0 because none of
the components are available to build: the test suite 'mostly-harmless-test'
is not available because the solver picked a plan that does not include the
test suites, perhaps because no such plan exists. To see the error message
explaining the problems with such plans, force the solver to include the test
suites for all packages, by adding the line 'tests: True' to the
'cabal.project.local' file.
```

but `cabal build -w ghc-9.2.1 --enable-tests` (as well as `cabal test -w ghc-9.2.1` with 'tests: True' in `cabal.project.local`) goes through fine and tests run. However, after removing `cabal.project.local`, `cabal test -w ghc-9.2.1` fails again the same.

The same attempts with ghc-8.10.7 work fine.

**Expected behavior**

The solver should pick the right plan without forcing. Forcing would ideally be used only to show the error that prevents the solver from finding a plan. That's the assumption under which ~@gelisam and~ me operated in #7834 and related tickets. Now it's all put into question.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.