elm-explorations / elm-explorations/test

Enable custom messages for test failures while still rendering equality messages

Open
#234 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Elm
Stars
244
Forks
40
Avg merge
1d 23h
Merged PRs (30d)
2

Description

(Here is an idea that I discussed with @Janiczek at Elm Camp 2024)

I've been working on `edkelly303/elm-exhaustive-generators`, a package that exhaustively generates Elm values, which might potentially be useful in tests.

As part of the API, I've included an `Exhaustive.test` function that works much like `Test.fuzz` from elm-test, except that it takes an `Exhaustive.Generator a` instead of a `Fuzz.Fuzzer a`.

Here's an example of a test that will fail:

```elm
module Example exposing (..)

import Exhaustive
import Expect
import Test exposing (..)

zeroToTen : Exhaustive.Generator Int
zeroToTen =
Exhaustive.values (List.range 0 11) -- oops, this should be `List.range 0 10`

suite : Test
suite =
only <|
Exhaustive.test zeroToTen "Input multiplied by 10 should never be larger than 100" <|
\input ->
input * 10 |> Expect.atMost 100
```

The output we get is:

```
Compiling > Starting tests

elm-test 0.19.1-revision12
--------------------------

Running 1 test. To reproduce these results, run: elm-test --fuzz 100 --seed 38488375388147

> Example
> Input multiplied by 10 should never be larger than 100

110

│ Expect.atMost

100

TEST RUN FAILED

Duration: 186 ms
Passed: 0
Failed: 1
```

What I'd really like is to be able to insert some text into the test output to log the generated value that was passed into the test, and the index of this value in the sequence of terms generated by the `Exhaustive.Generator`. For example, something like this:

```
Compiling > Starting tests

elm-test 0.19.1-revision12
--------------------------

Running 1 test. To reproduce these results, run: elm-test --fuzz 100 --seed 38488375388147

> Example
> Input multiplied by 10 should never be larger than 100

Test failed with generated value `11` (n = 11)

110

│ Expect.atMost

100

TEST RUN FAILED

Duration: 186 ms
Passed: 0
Failed: 1
```

In the current `elm-explorations/test` API, `Expect.onFail` _almost_ does what I'm looking for, but unfortunately the `String` that I provide as my custom error message completely replaces the default message, which means it no longer prints out this (very useful) bit of output:

```
110

│ Expect.atMost

100
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.