WebAssembly / WebAssembly/testsuite

Merging regression suites from engines

Open
#153 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
WebAssembly
Stars
249
Forks
65
Avg merge
33m
Merged PRs (30d)
5

Description

In some backroom discussions, I've raised the possibility of putting together the union of all the regression suites that are accumulating in various engines. For example, Wizard has thousands of .wast files that are either new tests, regression tests for old bugs, or expansions of spec tests. Due to the magic of .wast, all such tests pass on the Wasm 3.0 spec interpreter (yay!), so I think these could also be useful to other engines. Similarly, other engines' accumulated regression tests have been useful for Wizard as well.

I wonder if there is appetite for duplicating all of the .wast tests out there into this repo? We could do this manually at first, letting engine maintainers push new tests here via PRs. In the future we could also consider scripts to pull in said tests.

There are maybe some issues to discuss, such as making sure that tests mirrored or duplicated here actually do accurately represent the spec, but I think this could be good for the ecosystem.

Thoughts?

@fitzgen @eqrion @kmiller68 @tlively @cfallin

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing .wast regression suites in Wizard and other engines, along with how tests are currently represented in this repository. Check whether candidate mirrored tests accurately represent the Wasm 3.0 specification. Done means the project has an agreed approach for importing and maintaining the combined suites.

Written by the indexing model from the issue text.

Assessment

Tech stack
wasm
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.