rtfeldman / rtfeldman/node-test-runner

Quick way to debug failing fuzz tests

Open
#327 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
JavaScript
Stars
140
Forks
83
Avg merge
8h 25m
Merged PRs (30d)
5

Description

My workflow with fuzzy tests is something like

  1. I write fuzzy tests
  2. The fuzzy tests find some values that make the test fail
  3. To reproduce the error exactly, I manually replace the fuzzy test parameters with the specific values found by the fuzzy tests
  4. I run the offending code, usually with a lot of Debug.log to understand why it is failing
  5. I get swamped in thousands of Debug.log lines because the fuzz test is still being executed hundreds of time, and it takes a lot more time than necessary

Right now the best way I found around this is to cut & paste the whole code of the Fuzzy test and turn it into a new, non-fuzzy test that runs with the parameters found by the fuzzy test, and use Test.only to skip the other tests.

I don't know if the above is the best approach. If it isn't, it would be nice to have a way to run a fuzzy test in non-fuzzy mode, ie with specific parameters, without having to fiddle too much with the code.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The payload names no implementation file or test entry point. Start by tracing how fuzzy tests and Test.only are handled in node-test-runner, then determine how specific failing parameters could run a single case without duplicating the test. Done means those values can be replayed with normal Debug.log output while other fuzz executions are skipped.

Written by the indexing model from the issue text.

Assessment

Tech stack
elm, javascript
Domain
testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.