mcdc-project / mcdc-project/mcdc

Establish comprehensive code coverage and Coveralls reporting

Open
#489 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ci help wanted testing
Dominant language
Python
Stars
61
Forks
38
Avg merge
1d 13h
Merged PRs (30d)
10

Description

Summary

Establish a reliable code-coverage workflow for MC/DC, improve meaningful test coverage, and integrate reporting with Coveralls.

Current baseline

The merged unit-test report currently shows approximately:

  • 53% line coverage
  • 44% branch coverage
  • 52% overall coverage as calculated by Coverage.py

Regression tests are not included.

The unit suite runs in both Python and Numba modes. Their coverage data must be handled carefully because separate reports can overwrite each other, while Python coverage tools do not fully observe Numba-compiled execution.

Work plan

  • Establish the coverage workflow: Make coverage reproducible locally and in CI, with clear handling of multiple test modes and reports.
  • Review coverage gaps: Identify important uncovered behavior and prioritize work based on risk and value rather than percentage alone.
  • Improve tests: Add meaningful tests, strengthen existing tests, and create focused follow-up issues where useful.
  • Review generated code: Determine whether uncovered generated code, including code_factory output, should be tested, excluded, or removed.
  • Evaluate broader coverage: Assess how regression, MPI, and compiled execution modes should contribute without producing duplicate or misleading results.
  • Integrate Coveralls: Report coverage and coverage changes for pull requests and maintained branches, with an appropriate policy for preventing regressions.
  • Document the approach: Explain how coverage is produced, merged, interpreted, and intentionally limited.

Acceptance criteria

  • Coverage is reproducible locally and in CI.
  • High-priority coverage gaps have been reviewed and meaningfully improved.
  • Generated code and intentional exclusions are documented.
  • The contribution of unit, regression, Python, Numba, and MPI execution is understood.
  • Coveralls reports coverage and coverage changes.
  • Contributors can find instructions for running and interpreting coverage.
  • Remaining substantial work is tracked in follow-up issues.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing unit-test commands and CI configuration, then examine how Python and Numba coverage data are currently produced and combined. Review the reported gaps and generated code, including code_factory output, before deciding what should be tested or excluded. Done means reproducible local and CI coverage, documented interpretation and exclusions, meaningful high-priority tests, and Coveralls reporting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.