apache / apache/lucenenet

Task: Set up Test Coverage Report

Open
#741 0 comments 0 reactions 0 assignees View on GitHub
is:enhancement pri:low Project Infrastructure testability up-for-grabs
Dominant language
C#
Stars
2.4k
Forks
658
Avg merge
3d 5h
Merged PRs (30d)
9

Description

While adding more tests is probably not an activity we will be focusing on outside of custom components and untested APIs that had significant refactoring, having some insight as to which APIs carry more risk of change than others would be useful. Also, knowing the percentage of test coverage would help us to convey how good or bad the coverage is for a particular component.

There are 2 parts to this.

- [ ] Workflow Integration
- [ ] Mailing List Notification

## Workflow Integration

Thanks to @nikcio, we now know that [running the test coverage report requires about 3-4 hours](https://github.com/apache/lucenenet/pull/709#issuecomment-1288804221) of server time, perhaps more. If we use GitHub Actions, Apache [doesn't put a time limit on agents](https://infra.apache.org/github-actions-secrets.html).

Also, thanks to @nikcio, we know that [SonarCloud](https://sonarcloud.io/project/overview?id=apache_lucenenet) can do some of the heavy lifting as far as UI integration with the report. So, I think adding the test coverage to the [sonar.yml GitHub Workflow](https://github.com/apache/lucenenet/blob/10c24f2939e334a0690fc5436a81c7fbfcad97f4/.github/workflows/sonar.yml) on a schedule (perhaps daily, perhaps less) is probably our best way of getting these reports.

We should probably divide up the scan into 2 triggers (assuming SonarCloud can deal with this):

1. A push to master trigger that launches the SonarCloud scan only
2. A scheduled event trigger that runs both the test coverage and the SonarCloud scan

Perhaps there is some way to cache the last code coverage report so the push scans can just "inherit" the coverage report from the full scan.

GitHub Actions allows tasks/jobs to run conditionally based on the trigger that fired them, which would allow us to conditionally run the code coverage. See the [schedule trigger documentation](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule).

## Notification

We have already set up a [build mailing list](https://infra.apache.org/github-actions-secrets.html) to send notifications for nightly and release builds (and whether they passed/failed). We can send an email to that mailing list when `SonarCloud` finishes its task to notify us that the report is ready for viewing (only when running on a schedule - pushes would cause a lot of noise).

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/sonar.yml and the linked SonarCloud project, then review the referenced GitHub Actions schedule-trigger documentation. Determine how scheduled runs can generate coverage while push runs perform only the SonarCloud scan, and how completion can notify the existing build mailing list; done means both workflow checkboxes operate without adding notification noise to push scans.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, github-actions
Domain
ci-cd, observability, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.