bytecodealliance / bytecodealliance/wasm-tools

Don't skip tests we expect to fail, instead assert that they fail

Open
#1,031 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.8k
Forks
351
Avg merge
16h 57m
Merged PRs (30d)
38

Description

Right now we just skip generating a `#[test]` for any test file that we don't support yet for whatever reason. But this means that if we add support for something but forget to retest whether the test files are supported now, they will be turned off indefinitely until we remember to check.

Instead, we should consider emitting `#[test]`s that are marked `#[should_panic]` because we expect them to fail. Then if they pass, the test suite will fail until we adjust the test expectation.

https://github.com/bytecodealliance/wasm-tools/blob/8ce48202bd9ac64f70acc3d30ee86fb695d22d2c/tests/roundtrip.rs#L136

https://github.com/bytecodealliance/wasm-tools/blob/8ce48202bd9ac64f70acc3d30ee86fb695d22d2c/tests/roundtrip.rs#L56

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.