eclipse-score / eclipse-score/tooling

coverage: integration mode — scope external module roots (reference_integration)

Open
#445 1 comment 0 reactions 1 assignee Claimed by @dcalavrezo-qorix View on GitHub
Dominant language
Rust
Stars
8
Forks
37
Avg merge
2d 13h
Merged PRs (30d)
22

Description

## Motivation

`eclipse-score/reference_integration` measures coverage **across the integrated modules** (`@score_communication`, `@score_persistency`, `@score_orchestrator`, `@score_kyron`, …) at the `known_good.json` pins — today via one `rust_coverage_report` per module (`rust_coverage/BUILD`, driven by `scripts/quality_runners.py`), assembled into a hand-written step summary.

That flow is built on the `rust_coverage_report` rule removed in score_tooling 2.x, so the repo cannot upgrade score_tooling until the shared pipeline can do the same job. And the job is genuinely different from module CI: **one aggregated report over all modules, including coverage produced by the integration tests (FIT) that exercise modules together** — a signal no single module CI can produce.

## Problem

`@score_tooling//coverage` is deliberately **in-workspace only**; in an integration repo all production code is external, so the scope is empty apart from local showcases. Concretely:

| Component | In-workspace assumption |
|---|---|
| `coverage_scope.bzl` aspect | skips files/archives under `external/`; label check `str(target.label).startswith("@@")` excludes external targets |
| `reporter.py` `load_baseline_objects()` | resolves manifest entries against `_main` rlocations |
| `reporter.py` source resolution | `--path-equivalence=/proc/self/cwd/,` — `external//…` covmap paths don't resolve under the workspace root (sources live in the output base) |
| `merger.py` `get_object_files_from_manifest()` | skips every `external/` manifest entry to ignore the Rust toolchain's llvm binaries — would also skip external modules' **test binaries** |
| `justify.py` | scans `--source-root` (the workspace) for `COV_JUSTIFIED` markers |
| `coverage_summary.py` | directory rollup keyed by path prefix — external paths need repo-aware grouping |

## Proposal: opt-in "integration mode"

1. **Scope over external roots**: `score_coverage_scope(deps = ["@score_persistency//src/rust/rust_kvs", …], include_external_repos = ["score_persistency", …])` (exact API TBD). The aspect collects sources/archives from the listed repos (transitive, same test/mock exclusion by traversal), emitting allowlist entries in exec-root form (`external//…`) and baseline manifest entries with the correct rlocation prefix per repo.
2. **Reporter**: resolve sources through the **exec root** (which contains `external//…`), not the workspace root — for both llvm-cov HTML rendering and LCOV/HTML path relativization (report paths as `/`).
3. **Merger**: replace the blanket `external/` skip with a narrower rule (skip toolchain repos only — e.g. entries whose repo provides the llvm tools — or detect the `llvm-cov`/`llvm-profdata` basenames), so external modules' instrumented test binaries are collected.
4. **Justifications**: allow multiple `--source-root`s (workspace + listed external repo roots) so modules' own `COV_JUSTIFIED` markers and YAMLs are honored; the integration repo itself may carry none and gate on raw.
5. **Summary**: group the directory rollup by `` first for external files.
6. Fold in the `cc_binary` gap found in eclipse-score/time#186: `cc_binary` exposes no `CcInfo`, so its own srcs (`main.cpp`) are not collected — add a rule-kind branch analogous to the existing `CrateInfo` branch for `rust_binary` (collect srcs + the coverage-built executable as baseline object).
7. Integration test: extend `coverage/integration_tests/` with a second local module consumed via `local_path_override` and scoped externally, asserting external files appear (incl. exact 0% baselines) and in-workspace behavior is unchanged when the option is off.

## Sharp edges to settle in the design first

- Canonical repo names (`score_persistency+`) vs apparent names in allowlist/manifest/rlocation paths — must not leak Bazel-version-specific canonical forms into reports.
- Exec-root source availability: sandboxed builds don't materialize every source under `execroot/_main/external`; verify llvm-cov `show` can render external sources (fallback: resolve via the output base `external/` tree).
- Two configurations of the same external library (host vs exec) — the scope transition applies `collect_code_coverage`; confirm archives come from the coverage-instrumented configuration only.
- QNX/on-target remains out of scope (#427).

## Acceptance

- reference_integration replaces `rust_coverage/` with a `tools/coverage/` scope over the external module roots; `bazel coverage --config=llvm_cov` over the module tests + FIT produces one report with per-repo rollups, exact 0% for untested external files, and the run-page summary; score_tooling can be bumped to 2.x there.
- Existing consumers see no change unless they opt in.
- Default (non-integration) integration test still green.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.