redhat-developer / redhat-developer/rhdh-cli

SonarCloud reports 0% coverage despite tests being generated and run

Open
#203 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug ready-for-triage triaged
Dominant language
TypeScript
Stars
6
Forks
14
Avg merge
2d 14h
Merged PRs (30d)
25

Description

What happened

PR #202 added 205 lines of tests in src/commands/new/command.test.ts covering all three plugin types, validation, error cleanup, and interactive prompting. The CI checks job runs yarn backstage-cli repo test --coverage and the tests job runs yarn test (which expands to backstage-cli package test --coverage), so coverage data IS generated. However, SonarCloud's quality gate comment reported "0.0% Coverage on New Code" — and the quality gate still passed, providing false confidence about test coverage.

Investigation found three gaps:

  1. .sonarcloud.properties does not set sonar.javascript.lcov.reportPaths — SonarCloud has no idea where to find coverage files.
  2. No sonarsource/sonarcloud-github-action step exists in any CI workflow — SonarCloud is running in Automatic Analysis mode, which performs its own checkout and cannot access CI-generated coverage artifacts.
  3. Coverage output from backstage-cli package test --coverage (written to coverage/lcov.info) is never uploaded as a CI artifact.

This affects all PRs, not just #202 — coverage has likely been invisible to SonarCloud for the entire project history.

What could go better

Coverage visibility is a foundational quality signal. Without it, SonarCloud's quality gate is checking only duplication and static analysis — its coverage assessment is meaningless. This gap means:

  • Neither the review agent nor humans can cross-reference SonarCloud coverage data to assess test adequacy.
  • The quality gate provides false assurance ("passed" despite 0% coverage).
  • Coverage trends over time are not tracked, making regressions invisible.

Confidence: High. The .sonarcloud.properties file was directly inspected and contains only a CPD exclusion. The CI workflows were read and confirmed to have no SonarCloud scanner step. The --coverage flag in test scripts confirms coverage data is generated but never consumed.

This is a systemic repo-level issue, not specific to PR #202. It likely predates the fullsend integration.

Proposed change

Two changes needed in redhat-developer/rhdh-cli:

  1. Add coverage report path to .sonarcloud.properties:

    sonar.javascript.lcov.reportPaths=coverage/lcov.info
    
  2. Switch from Automatic Analysis to CI-based analysis by adding a SonarCloud scanner step to .github/workflows/pr.yaml after the test step (or uploading coverage as an artifact and running the scanner in a dependent job). Automatic Analysis cannot access CI-generated coverage files — CI-based analysis is required.

Alternatively, if staying with Automatic Analysis is preferred, the project should document that SonarCloud coverage data is not available and consider whether the quality gate should enforce a coverage threshold once CI-based analysis is enabled.

Validation criteria

After the fix: (1) SonarCloud reports non-zero coverage on PRs that add or modify tested code. (2) The sonar.javascript.lcov.reportPaths property is set in .sonarcloud.properties. (3) A SonarCloud scanner step exists in CI, or coverage is uploaded as an artifact consumed by the scanner. Verify on the next 2-3 PRs that include test changes.


Generated by retro agent from https://github.com/redhat-developer/rhdh-cli/pull/202

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .sonarcloud.properties and .github/workflows/pr.yaml, then inspect the existing test commands that produce coverage/lcov.info. Confirm how the workflow currently runs tests and how a SonarCloud scanner step can consume that report. Done means the report path is configured, CI-based analysis or artifact consumption is present, and a PR with tested changes shows non-zero SonarCloud coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.