rtfeldman / rtfeldman/node-test-runner
Quick way to debug failing fuzz tests
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 140
- Forks
- 83
- Avg merge
- 8h 25m
- Merged PRs (30d)
- 5
Description
My workflow with fuzzy tests is something like
- I write fuzzy tests
- The fuzzy tests find some values that make the test fail
- To reproduce the error exactly, I manually replace the fuzzy test parameters with the specific values found by the fuzzy tests
- I run the offending code, usually with a lot of
Debug.logto understand why it is failing - I get swamped in thousands of Debug.log lines because the fuzz test is still being executed hundreds of time, and it takes a lot more time than necessary
Right now the best way I found around this is to cut & paste the whole code of the Fuzzy test and turn it into a new, non-fuzzy test that runs with the parameters found by the fuzzy test, and use Test.only to skip the other tests.
I don't know if the above is the best approach. If it isn't, it would be nice to have a way to run a fuzzy test in non-fuzzy mode, ie with specific parameters, without having to fiddle too much with the code.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The payload names no implementation file or test entry point. Start by tracing how fuzzy tests and Test.only are handled in node-test-runner, then determine how specific failing parameters could run a single case without duplicating the test. Done means those values can be replayed with normal Debug.log output while other fuzz executions are skipped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elm, javascript
- Domain
- testing, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100