codecheckers / codecheckers/discussion

Levels of success for CODECHECK certificates

Open
#12 3 comments 0 reactions 0 assignees View on GitHub
the check
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

I took a more detailed look at [SciGen.Report](https://scigen.report/) today together with its founder @amorim-cjs. SciGen.Report has different levels of reproducibility, which might be interesting to capture in the `codecheck.yml` as proper metadata. The CODECHECK certificates do mentions this kind of information (and the summary in the `codecheck.yml` might too), but a structured proper field with a set of predefined values could be valuable to intergration and visulisation of search results, for example.

Here is a screenshot of the SciGen.Report options when I want to submit a review and the variants that are relevant to CODECHECK (considering that "negative" CODECHECK certificates are not something that will ever be published in the same way):

* Yes, within margin, all of it as written
* Yes, within margin, but extra information was needed (please clarify)
* Yes, but only partially (please specify)

![image](https://user-images.githubusercontent.com/1325054/101757442-7594d580-3ad7-11eb-84fd-147aee3a8abd.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the reproducibility levels listed for SciGen.Report with the information already present in CODECHECK certificates and the summary in codecheck.yml. Define how the proposed predefined values would be represented and how integrations or search-result visualisations would consume them; done means the metadata options and their intended meanings are agreed.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.