[Tracking] Add code coverage for storage CI check
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 115
Description
## Summary
Add code coverage reporting for storage CI pipelines. This will help track, aggregate, and render coverage results from all supported languages, ensuring actionable feedback and consistency for engineering teams. The table and tasks below describe the current state, required enhancements, and recommended steps per language. This change should also apply to CI pipelines for other services.
### Current Status
| Aspect | Java | .NET | Python | JavaScript | Go | Rust |
|---|---|---|---|---|---|---|
| **Coverage Tool** | JaCoCo | Coverlet | `pytest-cov` + `coverage.py` | Istanbul (via Vitest) | `go test -coverprofile` (built-in) | None |
| **Report Format** | JaCoCo XML + HTML | Cobertura XML | Cobertura XML | Cobertura XML + HTML | Go coverprofile (need conversion to cobertura) | N/A |
| **HTML Report Generation** | ✅ JaCoCo generates HTML | ✅ ReportGenerator | Need fixes (worked before) | ✅ Vitest Istanbul HTML | ❌ None | ❌ None |
| **Aggregation Method** | `generate_aggregate_pom.py` → `jacoco:report-aggregate` | ReportGenerator merges per-project Cobertura files | `create_coverage.py` combines `.coverage_*` files → `coverage combine` | Per-package only (no cross-package merge) | `gocovmerge` to merge per-module coverprofiles | `cargo llvm-cov --workspace` (native workspace aggregation) |
| **Azure DevOps Rendering** | ✅ Working | ✅ Working | Needs fix | ✅ Working | ❌ Not rendering | ❌ Not rendering |
### Estimated Effort and Complexity
| Language | Effort Estimate | Complexity |
|---|---|---|
| Python | 2 day | Low |
| JavaScript | Done | Medium |
| Go | 1 week | Medium (add format conversion + publish) |
| Rust | 2 week | High (build from scratch) |
---
## Task Breakdown
### Python
| Task | Description | Preferred Owner |
|---|---|---|
| Investigate DevOps coverage rendering | Run a CI build and confirm Azure DevOps "Code Coverage" tab populates. | Language team |
### JavaScript / TypeScript
| Task | Description | Preferred Owner | - DONE
|---|---|---|
| Enable coverage in CI test template | Add coverage collection and `PublishCodeCoverageResults` to `eng/pipelines/templates/jobs/ci.tests.yml`. | EngSys team |
| Add CI coverage matrix leg | Update matrix with `PublishCodeCoverage: true` like live test leg. | EngSys team |
| Adjust Vitest coverage condition | Update `vitest.shared.config.ts` to enable coverage for CI variable. | EngSys team |
| Handle browser coverage | Decide if browser Cobertura should be published in Azure DevOps. | |
| Validate rendering | End-to-end validation in CI. | |
### Go
| Task | Description | Preferred Owner |
|---|---|---|
| Merge coverage data | Use `gocovmerge` to merge per-module `coverage.txt` files. | Language team |
| Convert to Cobertura | Use `gocover-cobertura` to convert merged profiles. | Language team |
| Publish to pipeline | Add publish task. | |
| Add matrix leg / gating var | Enable via variable and matrix leg. | |
| Validate rendering | E2E validation in CI. | |
### Rust
| Task | Description | Preferred Owner |
|---|---|---|
| Coverage tool selection | Use `cargo-llvm-cov` for coverage. | Language team |
| Add coverage leg | Use workspace aggregation and Cobertura/HTML generation. | Language team |
| Publish results | Add publish task. | |
| Gate with matrix variable | Add gating and matrix entry. | |
| Validate rendering | E2E validation in CI. | |
---
**Notes:**
- Leverage existing patterns and CI templates (`eng/pipelines/templates`).
- HTML report generation should be enabled where possible for engineering consumption.
- Pipeline overhead should be limited to covered jobs using gating/matrix variables.
Contributor guide
Research direction
Start by reading the task tables and the named CI files under eng/pipelines/templates, especially jobs/ci.tests.yml and vitest.shared.config.ts. Review the existing aggregation references to generate_aggregate_pom.py and create_coverage.py, then run a relevant CI build. Done means supported language coverage is aggregated, published, rendered in Azure DevOps, and gated by the intended matrix variables.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp, go, java, javascript, python, rust, typescript
- Domain
- build-system, ci-cd, devops, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100