WebAssembly / WebAssembly/testsuite
Merging regression suites from engines
Nobody has claimed this yet.
- Dominant language
- WebAssembly
- Stars
- 249
- Forks
- 65
- Avg merge
- 33m
- Merged PRs (30d)
- 5
Description
In some backroom discussions, I've raised the possibility of putting together the union of all the regression suites that are accumulating in various engines. For example, Wizard has thousands of .wast files that are either new tests, regression tests for old bugs, or expansions of spec tests. Due to the magic of .wast, all such tests pass on the Wasm 3.0 spec interpreter (yay!), so I think these could also be useful to other engines. Similarly, other engines' accumulated regression tests have been useful for Wizard as well.
I wonder if there is appetite for duplicating all of the .wast tests out there into this repo? We could do this manually at first, letting engine maintainers push new tests here via PRs. In the future we could also consider scripts to pull in said tests.
There are maybe some issues to discuss, such as making sure that tests mirrored or duplicated here actually do accurately represent the spec, but I think this could be good for the ecosystem.
Thoughts?
@fitzgen @eqrion @kmiller68 @tlively @cfallin
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing .wast regression suites in Wizard and other engines, along with how tests are currently represented in this repository. Check whether candidate mirrored tests accurately represent the Wasm 3.0 specification. Done means the project has an agreed approach for importing and maintaining the combined suites.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- wasm
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100