rust-lang / rust-lang/rust

Coverage instrumentation loses information in macro-generated code

Open
#123,567 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

A-code-coverage A-proc-macros C-bug E-needs-mcve T-compiler
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

I tried this code (using thiserror v1.0.58):

use thiserror::Error;

#[derive(Error, Debug)]
pub enum E {
    #[error("{}", if *.0 { "a" } else { "b" })]
    X(bool)
}

#[cfg(test)]
mod tests {
    use super::E;

    #[test]
    fn test() {
        assert_eq!(E::X(true).to_string(), "a");
    }
}

I expected to see this happen: When running the test with cargo llvm-cov --text, the else arm should be marked as uncovered while the then arm should be shown as covered.

Instead, this happened: The code in the #[error] attribute does not have any coverage data. When run with cargo llvm-cov --text, this is the result:

$ cargo llvm-cov --text
[...]
    1|       |use thiserror::Error;
    2|       |
    3|      1|#[derive(Error, Debug)]
    4|       |pub enum E {
    5|       |    #[error("{}", if *.0 { "a" } else { "b" })]
    6|       |    X(bool)
    7|       |}
    8|       |
    9|       |#[cfg(test)]
   10|       |mod tests {
   11|       |    use super::E;
   12|       |
   13|       |    #[test]
   14|      1|    fn test() {
   15|      1|        assert_eq!(E::X(true).to_string(), "a");
   16|      1|    }
   17|       |}

Note that the proc-macro generated code is shown as covering line 3.

If the code is wrapped in a function (either a normal function or a lambda), the coverage information is correct:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum E {
    #[error("{}", (|| if *.0 { "a" } else { "b" })())]
    X(bool)
}

#[cfg(test)]
mod tests {
    use super::E;

    #[test]
    fn test() {
        assert_eq!(E::X(true).to_string(), "a");
    }
}

will lead to

cargo llvm-cov --text
[...]
    1|       |use thiserror::Error;
    2|       |
    3|      1|#[derive(Error, Debug)]
    4|       |pub enum E {
    5|      1|    #[error("{}", (|| if *.0 { "a" } else { "b" })())]
                                                          ^0
    6|       |    X(bool)
    7|       |}
    8|       |
    9|       |#[cfg(test)]
   10|       |mod tests {
   11|       |    use super::E;
   12|       |
   13|       |    #[test]
   14|      1|    fn test() {
   15|      1|        assert_eq!(E::X(true).to_string(), "a");
   16|      1|    }
   17|       |}

Note that now "b" is shown as uncovered.

I initially encountered this while writing a proc-macro similar to thiserror that generates error messages based on enum definitions. To work around this issue, I wrapped the code that I wanted to have coverage information in IIFEs inside the proc-macro.

Meta

rustc --version --verbose:

$ rustc --version --verbose
rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)
binary: rustc
commit-hash: 9d79cd5f79e75bd0d2083260271307ce9acd9081
commit-date: 2024-04-05
host: x86_64-unknown-linux-gnu
release: 1.79.0-nightly
LLVM version: 18.1.2
$ cargo llvm-cov --version
cargo-llvm-cov 0.6.9
$ grep 'name = "thiserror"' Cargo.lock -A1
name = "thiserror"
version = "1.0.58"

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Rust reproducer in the issue and run cargo llvm-cov --text using the direct expression and the closure-wrapped expression. Investigate how coverage instrumentation attributes proc-macro-generated code, and consider the issue resolved when the direct if expression reports its covered and uncovered branches like the closure version.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.