llrs / llrs/intermittent_failures

CRAN flavor might not match R version used

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
HTML
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The R version of the logs of checks might not match those of the flavor! (See below). Scanning one of each flavor cannot be generalized to all the packages as they might be using a different R version.

I sent an email to cran@r-project.org:

> Dear CRAN,
>
> I am interested in the R version or svn revision of the versions used in CRAN checks.
> I use this together with tools::CRAN_check_results() to identify packages with intermittent issues not related to dependencies or R changes. I collect the version information from a single package for all checks and extrapolate to all packages. All code can be found here https://llrs.github.io/intermittent_failures/
>
> However, I found that sometimes the flavor of the check does not match the version used. I don't know how often this happens as I haven't checked all the CRAN checks.
> For instance, https://www.r-project.org/nosvn/R.check/r-release-macos-arm64/A3-00check.html which according to the name should be checked with R-release 4.2.0 reports (I checked this last week):
> R Under development (unstable) (2022-03-20 r81946).
> While https://www.r-project.org/nosvn/R.check/r-release-macos-arm64/Hmisc-00check.html is:
> using R version 4.2.0 (2022-04-22)
>
> I tried downloading a mirror of the CRAN but I couldn't find the logs there.
> Is there a way to programmatically retrieve the R version, the date of the check and the revision and R version in case of the devel or patched versions of each check ?
>
> Many thanks for this great service you do with CRAN,
>
> Lluís

Response:

> The precise version for a check is indeed the one you can find in the
> logfile.
>
> Best,
> Uwe Ligges

So intermittent_failures needs to check all log files to ensure the R version used matches between days.
If the date of the log is also provided it might be useful to skip them when there isn't a new check.
It might be faster to download all the logs and parse the first line of text (or 12 of the html).

---
Incidentally this would also provide some estimation as how often are (public) checks performed on each flavor & package.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the intermittent_failures code that retrieves CRAN check results, then inspect the CRAN log URLs described in the issue. Compare the R version and check date across all relevant flavor/package logs, and verify that unchanged checks can be skipped and check frequency can be estimated.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.