WebAssembly / WebAssembly/spec
Spec tests that conflict with mature proposals
Nobody has claimed this yet.
- Dominant language
- WebAssembly
- Stars
- 3.5k
- Forks
- 539
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 13
Description
There are several spec tests that test that a given module is malformed, where that same module would be considered well-formed under some mature-but-not-yet-standard proposal. For example, there is a test that modules with multiple memories are malformed, but this test is invalidated by the multi-memory proposal, and indeed the multi-memory proposal adds tests in the other direction, testing that modules with multiple memories are well-formed.
These contradictory tests are a source of complexity for engine and tool developers when they implement mature proposals like multi-memory because they either must change their decoding behavior based on enabled features or must selectively skip tests to resolve conflicts. https://github.com/WebAssembly/tool-conventions/issues/233 discusses an instance of real-world toil caused by the former strategy, and of course the latter strategy is not ideal, either.
I propose that we reduce the complexity for implementers here by updating the spec tests to avoid conflicts with mature proposals. Note that we already have to update the spec tests this way once a proposal is finished and merged into the spec; I'm simply proposing that we remove the conflicts earlier.
Concretely, I propose that we add the following item to phase 3, where it is expected that engines and tools will implement the proposal and run into these conflicts:
During this phase, the following proceeds in parallel:
...
- Upstream spec tests are updated to avoid conflicts with the feature
Does this seem like a good idea?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the phase 3 process entry in meetings/process/phases.md and the existing spec tests for malformed modules, including the multi-memory examples described in the issue. Determine which mature-proposal tests conflict, then define the process and test updates needed so those conflicts are removed without waiting for proposal completion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- wasm
- Domain
- compilers, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100