codecov / codecov/codecov-action

Support for xcresult-based uploads in `v4`.

Open
#1,367 5 comments 4 reactions 1 assignee View on GitHub

@nora-shap is already working on this.

Since Feb 19, 2025.

Area: Report Upload bug codecov/codecov-action
Dominant language
Python
Stars
1.7k
Forks
260
Avg merge
1d 7h
Merged PRs (30d)
1

Description

Problem

We found that tokenless uploads using action v3 are essential broken since Friday (e.g., see https://github.com/StanfordBDHG/LLMonFHIR/actions/runs/8592479578/job/23631941264).
This encouraged to explore an upgrade to the v4 action using organization wide tokens (nice feature btw). However, we quickly found that the xcode and xcode_archive_path options were completely removed from v4 without clear instructions on how to migrate.

So our question is, what are the drop in replacements for the xcode and xcode_archive_path options with v4? Are xcresult-based coverage uploads still supported (if so, how?) or were they just silently removed?
We would highly appreciate respective migration guides or according documentation for the most prominent use cases as both are currently completely missing.

I tired my best to provide you the best possible overview of our current situation and hope you can provide helpful feedback and point us towards the next steps.

Current Workflow

To highlight our current workflow:

  • We have build and test n actions each producing a .xcresult file with collected code coverage.
  • All of these files are uploaded as GitHub artifacts
  • Once all steps completed successfully, a new GitHub Actions Job runs to collect all artifacts from previous build steps, merge all coverage reports using xcrun xcresulttool merge ... creating a new CodeCoverage.xcresult.
  • Previously we used the codecov action with xcode: true and xcode_archive_path: CodeCoverage.xcresult options to upload the coverage report. This step is seemingly not possible anymore.
What we've tried

I tried to explore how such interactions are handled with v4. CLI options seem to mention some form of xcode plugin to be available (see https://docs.codecov.com/docs/cli-options#do-upload). The Supported Coverage Report Formats also still seem to mention some txt-based Xcode code coverage format. Also, when running codecovcli locally it randomly pulls code coverage reports from all, completely unrelated Xcode projects in derived data into the current directory (Not sure if that is the intended new way?). This doesn't work with our current workflow, as we don't have the derived data available (also we don't use the global derrivedData directory), as we have a dedicated step to assemble a self-contained xcresult package.

Attempts that failed

I have tried to manually convert the code coverage report of a xcresult package into a format that codecov understands. Simple attempts by just manually pulling out the coverage format in json or txt have failed (e.g., using the xcrun xccov view --report --json CodeCoverage.xcresult > cov.json). While the overall upload action completed successfully (by supplying the file argument with the respective output), the UI shows the error null. Not super helpful. See https://app.codecov.io/github/StanfordSpezi/SpeziFileFormats/commit/0a7434be32dea2e44e126f314ab9b116ec042e39.

CC: @PSchmiedmayer @philippzagar @vishnuravi

### Tasks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.