More detail/documentation for "functions" coverage
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 170
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 4
Description
This is a bit of a wishy washy issue, but it comes down to: I saw a big red number on my otherwise-green coverage report, but I still don't really understand what it means or how to get more detail on it (or how to fix it). As I go into more detail I've flagged the more specific questions and issues I have.
There's an example project here, and here's the coverage report. It's from a real Rust project I'm working on, it's deliberately not minimal (I'll get to that), and the command to generate the coverage report is:
env \
LLVM_PROFILE_FILE="test-binary-%p-%m.profraw" \
RUSTFLAGS="-C instrument-coverage" \
RUST_BACKTRACE=full \
cargo test
grcov . \
-t html \
--binary-path ./target/debug -s . \
--branch --llvm \
--ignore-not-existing --keep-only 'src/**' --ignore 'src/bin/test_*' \
--excl-br-start "mod tests \{" --excl-start "mod tests \{" \
--excl-br-line "#\[derive\(" --excl-line "#\[derive\(" \
-o coverage
When I generate the report, it shows the line and branch coverage as 100%. But between those two is something labelled "functions" which sits at 52.27% (in RED). Now, with no other information, I'd guess that that means only about 50% of the functions are being tested. But several things make me think this guess is wrong:
- How could line coverage be 100% if function coverage is only 50%?
- Drilling down into the code view shows every line to be green. (Question: I assume this is because that view only shows line coverage? Or not? Is there a way to switch between the three metrics?)
- I did search for questions/issues around this, and found #476 and $454. But both those issues say to try the "new" (in 2020) source-based coverage. I'm pretty sure I'm using that now. (Right?)
- Those issues say that this might be related to macros. But (issue:) the report does not show me what macros, either by highlighting them in the place they're used or by listing them separately.
- This is why I haven't made a minimal example: if I reduce this project down to a single macro, it's easy to see the problem with grcov's analysis but it's also hard to see the problem with its reporting. There it is! I might say. That's the macro causing all the trouble! But try doing that with this project and you'll quickly run out of patience.
Issue: Documentation would remove the guesswork, but I cannot find details on this in Grcov's docs.
My overall issue is simply this: if low function coverage warrants a big, red number on the top of the report, it should also provide a way to account for that number, near to where it's shown. A link to documentation would be better than nothing, but ideally a way to visually display why the number is what it is.
Environment details
Gitlab CI: rust:1-slim docker image
$ cargo --version
cargo 1.67.1 (8ecd4f20a 2023-01-10)
$ rustc --version
rustc 1.67.1 (d5a82bbd2 2023-02-07)
$ grcov --version
grcov 0.8.13
Dev machine: Ubuntu 22.10
~ ⚬ cargo --version
cargo 1.67.1 (8ecd4f20a 2023-01-10)
~ ⚬ rustc --version
rustc 1.67.1 (d5a82bbd2 2023-02-07)
~ ⚬ grcov --version
grcov 0.8.13
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked grcov-example project, its generated coverage report, and the documented command using --branch and --llvm. Clarify what the functions, lines, and branches metrics mean, how macros affect the report, and what documentation or report links would let users account for the displayed function coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- documentation, testing-qa
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100