stdlib-js / stdlib-js/stdlib

[RFC]: Update `@stdlib/assert/*` benchmarks to measure affirmative/negative test values

Open
#1,148 12 comments 0 reactions 0 assignees View on GitHub
Accepted difficulty: 1 Good First Issue Help Wanted JavaScript priority: Low RFC
Dominant language
JavaScript
Stars
6k
Forks
1.3k
Avg merge
1d 3h
Merged PRs (30d)
611

Description

Many of the benchmarks in [`@stdlib/assert/*`](https://github.com/stdlib-js/stdlib/blob/develop/lib/node_modules/%40stdlib/assert) measure performance of an "average" test case, meaning that test values are a mixture of values resulting in `true` and `false` values.

An example package having such benchmarks is [`@stdlib/assert/is-string`](https://github.com/stdlib-js/stdlib/blob/82ee4a5e90ee55bbb01c0977151ac5e8b25fbf10/lib/node_modules/%40stdlib/assert/is-string/benchmark/benchmark.js).

Preferably, we'd split benchmarks in separate positive/negative benchmark runs to have a better handle on performance depending on the input value type. While, for many assert packages, the difference is not likely to be significant, for others, the performance difference will be significant, as the assertion logic is more complex or may, in the attempt of robustness, perform various operations which are slow, such as walking the prototype chain or resolving a constructor name, etc. In these instances, having a more granular understanding of performance is useful, as downstream consumers can then make accommodations if such perf cliffs are undesired (e.g., when implementing dispatch logic and needing to resolve an input value's "type" before delegating to a specialized implementation).

An example package having benchmarks which are split into positive/negative test values is
[`@stdlib/assert/is-complex-like`](https://github.com/stdlib-js/stdlib/blob/develop/lib/node_modules/%40stdlib/assert/is-complex-like/benchmark/benchmark.js).

The request in this issue is to refactor any assertion packages which are not split into positive/negative test values to be similar to [`@stdlib/assert/is-complex-like`](https://github.com/stdlib-js/stdlib/blob/develop/lib/node_modules/%40stdlib/assert/is-complex-like/benchmark/benchmark.js).

Contributor guide

Open the contributing guide

Research direction

Compare the benchmark/benchmark.js files for @stdlib/assert/is-string and @stdlib/assert/is-complex-like to identify assertion packages whose inputs are still mixed. Refactor the applicable benchmarks to separate affirmative and negative test values, then run the relevant benchmark checks to confirm both cases are measured independently.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.