elm-explorations / elm-explorations/test
Enable custom messages for test failures while still rendering equality messages
- Dominant language
- Elm
- Stars
- 244
- Forks
- 40
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 2
Description
(Here is an idea that I discussed with @Janiczek at Elm Camp 2024)
I've been working on `edkelly303/elm-exhaustive-generators`, a package that exhaustively generates Elm values, which might potentially be useful in tests.
As part of the API, I've included an `Exhaustive.test` function that works much like `Test.fuzz` from elm-test, except that it takes an `Exhaustive.Generator a` instead of a `Fuzz.Fuzzer a`.
Here's an example of a test that will fail:
```elm
module Example exposing (..)
import Exhaustive
import Expect
import Test exposing (..)
zeroToTen : Exhaustive.Generator Int
zeroToTen =
Exhaustive.values (List.range 0 11) -- oops, this should be `List.range 0 10`
suite : Test
suite =
only <|
Exhaustive.test zeroToTen "Input multiplied by 10 should never be larger than 100" <|
\input ->
input * 10 |> Expect.atMost 100
```
The output we get is:
```
Compiling > Starting tests
elm-test 0.19.1-revision12
--------------------------
Running 1 test. To reproduce these results, run: elm-test --fuzz 100 --seed 38488375388147
> Example
> Input multiplied by 10 should never be larger than 100
110
╷
│ Expect.atMost
╵
100
TEST RUN FAILED
Duration: 186 ms
Passed: 0
Failed: 1
```
What I'd really like is to be able to insert some text into the test output to log the generated value that was passed into the test, and the index of this value in the sequence of terms generated by the `Exhaustive.Generator`. For example, something like this:
```
Compiling > Starting tests
elm-test 0.19.1-revision12
--------------------------
Running 1 test. To reproduce these results, run: elm-test --fuzz 100 --seed 38488375388147
> Example
> Input multiplied by 10 should never be larger than 100
Test failed with generated value `11` (n = 11)
110
╷
│ Expect.atMost
╵
100
TEST RUN FAILED
Duration: 186 ms
Passed: 0
Failed: 1
```
In the current `elm-explorations/test` API, `Expect.onFail` _almost_ does what I'm looking for, but unfortunately the `String` that I provide as my custom error message completely replaces the default message, which means it no longer prints out this (very useful) bit of output:
```
110
╷
│ Expect.atMost
╵
100
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.