codecheckers / codecheckers/discussion

Scope of CODECHECK Makefile

Open
#19 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The [CODECHECK guidelines](https://codecheck.org.uk/guide/community-process#codechecker-tasks) state:
> Write a Makefile to re-run the workflow based on provided documentation, i.e., recreate all files listed in the manifest by runnign the command make codecheck. This target should run the whole or most suitable subset of the workflow and create the report.

But looking at previous CODECHECK examples, this usually does not seem to be the case, the Makefile only recreates the report based on the stored output files (as a side note: `make clean` still cleans the output files). For example in the last two codechecks: https://github.com/codecheckers/LICD_article and https://github.com/codecheckers/driftage .

As a more general question: what is the advantage of actually using a Makefile instead of e.g. a simple shell script or just a line stating what to do in the README file? Not to be misunderstood: I'm definitely a fan of Makefiles in general, since they are great for keeping track of changed dependencies, etc., but is this actually a CODECHECK use case? This touches on questions of the reproduciblity of the code check report itself. For example, if I run make in the `codecheck` directory in one of the above mentioned repositories, I will get a new `codecheck.pdf` which is actually wrong in a sense, because it will dynamically include *my* R session info, even though this is not the environment that has been used to generate the output files included in the document. I understand that the PDF is the "frozen" document that states everything in a non-ambiguous way, but the whole Makefile + dynamic Rmarkdown setup to me gives the wrong impression that the codecheck itself could be reproduced. This would only be the case if either the Makefile runs the code separately from the Rmarkdown file (which seems to be the case in at least one repository: https://github.com/codecheckers/Hopfield-1982/), or if the Rmarkdown included the actual computations (which is not feasible in general, I guess).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.