haskell / haskell/cabal

New field on library: `test-only: Bool`

Open
#10,900 29 comments 3 reactions 0 assignees View on GitHub
type: enhancement
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

# UPDATE

We seem to be converging to this plan:

- Libraries get a new field: `test-only: Bool`, by default False.
- Libraries with `test-only: True` can only be imported by other libraries with `test-only: True` or test-suites.
- Local libraries with `test-only: True` will only be enabled if `--enable-tests` is on, even if no other component depends on them. They will respect the usual component naming (i.e. `all` will build them if they are enabled).

There is however the worry of this resulting in more rebuild of the project or build-plans changing when enabling these libraries. This is _not_ a new problem, but something that already happens with test-suites (and benchmarks).

# Original

**Describe the feature request**

Before trying to take a stab at implementing it, I wanted to gather some feedback and opinions.

Making use of sublibraries, a Haskell package can define a sublibrary that is intended to be used only in tests, either from this package tests or by other package tests, but it is not intended to be consumed in "production code".

> For a precise example, we make use of testlibs in our code (https://github.com/IntersectMBO/ouroboros-consensus, look in the cabal files for `unstable-*-testlib` sublibraries). We would like to not track those in the version number nor reflect changes to them in the changelog to reduce the bureaucratic side of the code.

> Moreover, as our project has multiple packages, we need such sublibraries in order to share code between test-suites in different packages.

With the current mechanisms, there is no way to enable such a library only if tests are enabled. I think having the ability to do so would be very useful.

~The proposal I have is to define a new stanza `testlib` (or `test-library`). Such libraries will only be available if the building of tests is enabled.~

EDIT: The proposal is in fact that testlibs can only be depended on from test-suites or other testlibs. This also implies not building with `--enable-tests` won't possibly build the testlibs.

This achieves also an additional nice outcome: Packages that accidentally depend on some code that upstream dependencies thought of as just intended for testing, will fail to build if tests are disabled, correctly conveying the message that such function should not be used in production.

This mechanism can be emulated with cabal flags that expose or hide sublibraries (we used to have a flag `expose-sublibs` that did this) but this changes the API of a package, and as downstream components cannot specify cabal flags in their `build-depends`, the same problem cascades down: they have to also disable their testlibs with some trick or compilation of their testlibs will fail.

So I'm open to suggestions on how to best approach this, or whether there is already some solution that I'm not aware of.

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are named. Start by locating the existing handling for sublibraries, test-suites, benchmarks, and --enable-tests, then trace how local library dependencies and component naming are processed. Done means test-only libraries follow the stated dependency and enablement rules without breaking build-plan behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.