redhat-developer / redhat-developer/rhdh-cli
SonarCloud reports 0% coverage despite tests being generated and run
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 6
- Forks
- 14
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 25
Description
What happened
PR #202 added 205 lines of tests in src/commands/new/command.test.ts covering all three plugin types, validation, error cleanup, and interactive prompting. The CI checks job runs yarn backstage-cli repo test --coverage and the tests job runs yarn test (which expands to backstage-cli package test --coverage), so coverage data IS generated. However, SonarCloud's quality gate comment reported "0.0% Coverage on New Code" — and the quality gate still passed, providing false confidence about test coverage.
Investigation found three gaps:
.sonarcloud.propertiesdoes not setsonar.javascript.lcov.reportPaths— SonarCloud has no idea where to find coverage files.- No
sonarsource/sonarcloud-github-actionstep exists in any CI workflow — SonarCloud is running in Automatic Analysis mode, which performs its own checkout and cannot access CI-generated coverage artifacts. - Coverage output from
backstage-cli package test --coverage(written tocoverage/lcov.info) is never uploaded as a CI artifact.
This affects all PRs, not just #202 — coverage has likely been invisible to SonarCloud for the entire project history.
What could go better
Coverage visibility is a foundational quality signal. Without it, SonarCloud's quality gate is checking only duplication and static analysis — its coverage assessment is meaningless. This gap means:
- Neither the review agent nor humans can cross-reference SonarCloud coverage data to assess test adequacy.
- The quality gate provides false assurance ("passed" despite 0% coverage).
- Coverage trends over time are not tracked, making regressions invisible.
Confidence: High. The .sonarcloud.properties file was directly inspected and contains only a CPD exclusion. The CI workflows were read and confirmed to have no SonarCloud scanner step. The --coverage flag in test scripts confirms coverage data is generated but never consumed.
This is a systemic repo-level issue, not specific to PR #202. It likely predates the fullsend integration.
Proposed change
Two changes needed in redhat-developer/rhdh-cli:
-
Add coverage report path to
.sonarcloud.properties:sonar.javascript.lcov.reportPaths=coverage/lcov.info -
Switch from Automatic Analysis to CI-based analysis by adding a SonarCloud scanner step to
.github/workflows/pr.yamlafter the test step (or uploading coverage as an artifact and running the scanner in a dependent job). Automatic Analysis cannot access CI-generated coverage files — CI-based analysis is required.
Alternatively, if staying with Automatic Analysis is preferred, the project should document that SonarCloud coverage data is not available and consider whether the quality gate should enforce a coverage threshold once CI-based analysis is enabled.
Validation criteria
After the fix: (1) SonarCloud reports non-zero coverage on PRs that add or modify tested code. (2) The sonar.javascript.lcov.reportPaths property is set in .sonarcloud.properties. (3) A SonarCloud scanner step exists in CI, or coverage is uploaded as an artifact consumed by the scanner. Verify on the next 2-3 PRs that include test changes.
Generated by retro agent from https://github.com/redhat-developer/rhdh-cli/pull/202
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .sonarcloud.properties and .github/workflows/pr.yaml, then inspect the existing test commands that produce coverage/lcov.info. Confirm how the workflow currently runs tests and how a SonarCloud scanner step can consume that report. Done means the report path is configured, CI-based analysis or artifact consumption is present, and a PR with tested changes shows non-zero SonarCloud coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, typescript
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100