easystats / easystats/easystats
Codecov uploads are failing silently in the shared test-coverage workflow
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 1.2k
- Forks
- 88
- PR merge metrics
- No merged PRs in 30d
Description
Following up on the stale coverage badges from easystats/easystats#477. The badges themselves are fine, the data behind them just stopped arriving a while ago: Codecov's last update is 2024-10-21 for correlation, 2025-10-05 for insight, 2025-11-13 for parameters.
What's happening is that the uploads get rejected and the job doesn't notice. correlation's July 25 run on main is a clean example. It finished green, and this is how it ended:
Request failed [429]. Retrying in 2.9 seconds...
[1] "Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 2120s."
The next line is "Post job cleanup". covr::codecov() prints the error response but never calls stop_for_status() on a non-2xx, so the step exits 0 and the run stays green while nothing lands. Same story in the August 9 PR run.
So it's a rate limit rather than an outright rejection: tokenless uploads through the legacy /upload/v2 endpoint all share one global bucket, which Codecov's docs describe as unreliable by design.
A CODECOV_TOKEN gets us out of that bucket, but I don't think adding the env var here is enough on its own. This is a reusable workflow, and GitHub only passes GITHUB_TOKEN into those automatically, not org or repo secrets. So the token would have to exist as an org secret, each caller would have to pass it (either a named workflow_call secret or secrets: inherit), and then it gets read here. Three places, and the first one needs owner rights I don't have.
The other direction is to drop the legacy endpoint entirely: have covr write cobertura.xml and hand that to codecov/codecov-action, which is what the current r-lib/actions example does. Bigger diff, but it can fail loudly, and a silent failure is really what let this sit unnoticed for so long.
Separately, while I was digging: insight's test-coverage hasn't succeeded on main since 2025-12-08, so for that package nothing reaches the upload step at all. Every run since dies the same way:
Error: running the tests in 'testthat.R' failed
Error: Failure in `.../insight-tests/testthat.Rout.fail`
That's all we get, because the workflow neither prints nor uploads testthat.Rout.fail (those runs have zero artifacts), so the actual test failure is invisible. The r-lib/actions example has steps for both. Looks like its own problem, happy to open a separate issue if you'd rather track it there.
Happy to write the PR for whichever direction you prefer! The token route just needs someone with owner rights to create the secret first.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the shared test-coverage workflow, its reusable workflow inputs, and the July 25 correlation run on main. Compare the current covr::codecov() path with the r-lib/actions example, then verify that rejected uploads fail the job and that insight's testthat.Rout.fail is exposed as an artifact when tests fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, r, yaml
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100