UnitTestBot / UnitTestBot/usvm

[TS PBT] Replay and shrink USVM counterexamples through PBT backends

Open
#353 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Kotlin
Stars
33
Forks
27
Avg merge
4d 3h
Merged PRs (30d)
15

Description

Part of #345. Depends on #352 and #384.

Goal

Confirm USVM candidate counterexamples in the original TypeScript runtime and reuse the concrete backend's shrinking.

Scope

  • Consume the candidate inputs already extracted by #352 through JsConcreteValue. Do not add another symbolic-value decoder or Kotlin-to-Node process protocol.
  • Reuse the existing backend execution/reproduction path and the shared invocation contract from #384.
  • Check membership in the declared input domain, run the precondition, and then the original predicate.
  • Classify each replay as confirmed violation, property holds, domain/precondition rejection, unrepresentable input, unsupported replay, or replay error. A thrown precondition is an error; a thrown predicate follows #384.
  • Preserve undefined, null, NaN, infinities, negative zero and supported tuple/array structure without coercion.
  • For fast-check, feed a confirmed witness through its existing explicit-example/reproduction support and verify that shrinking actually runs for supported domains. An example being executed is not proof it was minimized.
  • Recheck a minimized result against the same domain, precondition and predicate. If shrinking is unsupported, fails, or exhausts its budget, retain the original confirmed witness and record the limitation.
  • Store the original candidate, replay classification, minimized witness when available, actual backend version and existing reproduction data.

Definition of Done

  • Confirmed, spurious, rejected, throwing, unrepresentable and shrinkable fixtures pass through the real TypeScript runtime.
  • A minimized witness remains inside the declared domain and still violates the same property.
  • Aliases and mutation isolation follow #384 during explicit examples, replay and shrinking.
  • A backend without shrinking can still confirm and report a counterexample.
  • Replay/shrinking share the run deadline and never turn timeout/tool failure into a property failure.
  • No unconfirmed symbolic result is published as a final property counterexample.
  • Implementation and focused tests reuse the existing adapter and codecs.

A new concrete EtsIR interpreter, general backend capability framework, or custom shrinker is not required.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the candidate inputs from #352, JsConcreteValue, and the shared invocation contract from #384; trace them through the existing backend execution and reproduction path. Add focused tests using the existing adapters and codecs for replay classifications, explicit examples, and shrinking. Done means real TypeScript-runtime fixtures preserve values and aliases, minimized witnesses still violate the property, and unsupported or timed-out shrinking is reported without publishing unconfirmed counterexamples.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin, typescript
Domain
devtools, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.