hermit-os / hermit-os/kernel

Add code coverage (LLVM source based coverage)

Open
#148 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
1.5k
Forks
132
Avg merge
1d 14h
Merged PRs (30d)
39

Description

This is something I've wanted to do for some time, but I haven't had the time.
I'll use this issue to write down some of the issues that need to be solved.
If someone has the time to address this feel free to open a PR.

Source-based coverage support via LLVM [landed in the rust compiler](https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html) somewhat recently.
It can be enabled via the nightly RUSTFLAG [`-Zinstrument-coverage`](https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html)

Challenges specific to rusty-hermit:

## RUSTC_WRAPPER

RUSTFLAGS apply to everything, including dependencies, build scripts and macros.
A [RUSTC_WRAPPER](https://doc.rust-lang.org/cargo/reference/config.html#buildrustc-wrapper) program should be written that filters the RUSTFLAG `-Zinstrument-coverage`
flag and only applies it to relevant crates.
In particular crates such as `core` _must_ be excluded, since `profiler_builtins`,
which is required for `-Zinstrument_coverage` relies on `core`.

It should be evaluated if we can include `std`, since we provide a hermit specific `std`
library, and coverage metrics for that would be useful as well.

### Optimizations

Optimizations are detrimental to code coverage so they should be disabled, i.e.
by at least using the flags `-Copt-level=0 -Clink-dead-code`.
If possible we should again use the RUSTC_WRAPPER to control this, so that
crates without code coverage are optimized, while crates with code coverage are not.

Previous tests have shown that rusty-hermit may be lacking some (mathematical) functions, which
are never called. `-Clink-dead-code` might surface such problems.

## Profiler_builtins

This can be solved independently from the previous issue via the following MWE:

In rusty-hermit add `profiler_builtins` to the build-std configuration `.cargo/config`.
Run `cargo rustc -p rusty_demo -- -Zinstrument-coverage`. This is a sort of poor mans
RUSTC_WRAPPER where the `-Zinstrument-coverage`flag is only passed to the rusty_demo crate.
Of course this means that the actual kernel is not instrumented, but i believe instrumenting
the userspace application first is easier.

This will probably result in a linker error where some `__llvm_profile_*` functions are missing.
The issue is that the llvm source code files for the profiler builtins are not shipped with the nightly
and the build script (see [`profiler_builtins`](https://github.com/rust-lang/rust/tree/master/library/profiler_builtins) ) in the rust compiler silently ignores the missing files and simply does nothing.

To test things out you can visit [rust/src](https://github.com/rust-lang/rust/tree/master/src)
and clone the linked version of the llvm-project.
You can then copy the compiler-rt folder into "$HOME/.rustup/toolchains/nightly-2020-12-23-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/llvm-project/".
Replace the version in the above string to whatever version you are using (could also be simply nightly) and make sure that rustup also installed the `rustsrc` component.
Opening an issue on rustup asking for a profiling component which installs the necessary files
could also be an option.

Now when building the application (with verbose option) you should get new linker errors, that mention
undefined symbols such as `strlen`, `fopen` etc. originating from the llvm profiler c files.

To solve that I'd take a look at [hermit playground](https://github.com/hermitcore/hermit-playground), but additional steps might be needed, since in the end we need the symbols in the kernel code too, if we want coverage there.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.