nasa / nasa/CF

Remove use of random numbers in coverage test

Open
#86 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

unit-test
Dominant language
C
Stars
120
Forks
77
Avg merge
7d 18h
Merged PRs (30d)
3

Description

The objective of a coverage test is to exercise the various paths in the implementation to ensure they do not trigger any undefined behavior. It is not intended to be a "fuzz test" - although fuzz testing can be valuable, that is a separate test, typically done using a standard (non-stub) build via the public interface, where arbitrary/random input values are expected to be fully scrubbed and handled correctly - not for internal APIs that are expected to have more controlled inputs.

Currently the CF unit tests use lots of random values, and in particular may pass values that are not even within acceptable range, or do not make sense when paired with other inputs (e.g. a pointer and size where the size is a random value, exceeding the actual size of the object being pointed to). This randomness does not really add value to the coverage test objective, if anything it detracts from it by (possibly) creating opportunities for test cases to follow different paths through the code under test with each invocation.

The unit tests for CF should be scrubbed to remove use of random values, replace with a single/specific value(s) that are guaranteed to follow the intended path through the code under test.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the CF unit tests and identifying their uses of random values. Replace those values with specific inputs that follow the intended implementation paths, then run the unit-test suite to confirm the tests remain passing and deterministic.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.