bytecodealliance / bytecodealliance/wasmtime
cranelift: line coverage gaps in ISLE-generated lowering
- Dominant language
- Rust
- Stars
- 18.6k
- Forks
- 1.8k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 126
Description
I was musing about whether it would be possible for us to generate inputs to ISLE lowering that would exercise all code paths. So it occurred to me to check what coverage the cranelift unit/integration tests already have.
The line coverage reports suggest high coverage, but there are some gaps that might be worth discussion.
## Coverage Generation
I found the [`taiki-e/cargo-llvm-cov`](https://github.com/taiki-e/cargo-llvm-cov) tool to be very easy to use for coverage report generation.
In the `cranelift` directory:
```
cargo llvm-cov --disable-default-ignore-filename-regex --ignore-filename-regex='cargo/' --html
```
By default the `llvm-cov` tool won't report coverage for the ISLE generated files. The `--disable-default-ignore-filename-regex` was necessary to disable this default behavior, and then `--ignore-filename-regex='cargo/'` ignores coverage from external dependencies.
## Results
High-level coverage metrics for the ISLE generated files (columns are function/line/region):

There's a lot of uncovered code to sift through, a lot of which is clearly not a concern (including uncovered `unreachable!` statements). But there seem to be some cases in here that might be worth looking into. For example, in `constructor_to_amode_add`:

## Questions
* Are any of these coverage gaps interesting, and worth adding tests for?
* Are these gaps covered by fuzzing? (related #6293)
* Is 100% line coverage actually a reasonable goal here? What is the right metric? (related #1151)
cc @jameysharp @fitzgen
Contributor guide
Assessment
This issue has not been assessed yet.