hedgehogqa / hedgehogqa/fsharp-hedgehog

Run using the same seed until all tests pass

Open
#424 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
F#
Stars
284
Forks
31
PR merge metrics
No merged PRs in 30d

Description

I came across a blog post on property testing that used [jqwik](https://jqwik.net/). It seems to have some nice functionality that would be great to have in Hedgehog.

One useful feature is [automatically rerunning falsified properties](https://jqwik.net/docs/current/user-guide.html#rerunning-falsified-properties): If a property fails, jqwik by default uses the same seed for that property until it passes. It would be great to have this functionality in Hedgehog. The current behavior requires me to change `check` to `recheck` with the printed seed value for all failing properties to be sure that I am exposing the same failing behavior, and then change all the properties back after they pass. This is inconvenient.

I'm not sure how this can best be implemented, but I guess some temporary disk storage is needed for persistence between runs. Perhaps a temp folder with files named using a SHA256 hash of information that identifies the property (while being as robust as reasonably possible against changes to the source code), with the file contents being the seed value. If a property fails, the file is saved. If it passes, it is deleted (if it exists). Whenever a property is run, a seed is taken from the matching file, if it exists.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.