bytecodealliance / bytecodealliance/wasm-tools
Don't skip tests we expect to fail, instead assert that they fail
- Dominant language
- Rust
- Stars
- 1.8k
- Forks
- 351
- Avg merge
- 16h 57m
- Merged PRs (30d)
- 38
Description
Right now we just skip generating a `#[test]` for any test file that we don't support yet for whatever reason. But this means that if we add support for something but forget to retest whether the test files are supported now, they will be turned off indefinitely until we remember to check.
Instead, we should consider emitting `#[test]`s that are marked `#[should_panic]` because we expect them to fail. Then if they pass, the test suite will fail until we adjust the test expectation.
https://github.com/bytecodealliance/wasm-tools/blob/8ce48202bd9ac64f70acc3d30ee86fb695d22d2c/tests/roundtrip.rs#L136
https://github.com/bytecodealliance/wasm-tools/blob/8ce48202bd9ac64f70acc3d30ee86fb695d22d2c/tests/roundtrip.rs#L56
Contributor guide
Assessment
This issue has not been assessed yet.