forrtproject / forrtproject/replicatethis
[Nomination]: C2PA provenance labels and trust in digital news
- Dominant language
- Nunjucks
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
### DOI
10.1609/icwsm.v20i1.42749
### Journal
Proceedings of the International AAAI Conference on Web and Social Media (ICWSM), 20(1), 2267–2279
### Discipline
Computer Science
### Replication vs Reproduction
Replication
### Scientific Justification
This paper reports a practically important cross-country result: presenting C2PA provenance metadata improved perceived image transparency and credibility and increased trust in the news source. The online experiment included 6,114 participants reflecting audiences of six major news sources in the United States, United Kingdom, and Norway. Independent confirmation would be valuable because provenance labels are increasingly deployed as a trust intervention, while their effects may depend on label detail, wording, country, source, interface context, and participants' understanding of what C2PA does and does not establish.
A preregistered replication with a new sample would test whether the reported effects can be recovered in an independent implementation and would estimate their practical magnitude and heterogeneity. This nomination treats the original result as influential and methodologically interesting; it is not an assertion that the study is flawed. A public call discussing this target and the need for a scope check is also available at https://github.com/ReScience/call-for-replication/issues/9.
### Data Location (if Reproduction)
This nomination proposes a replication with new data. The article's preprint has an OSF DOI (https://doi.org/10.31219/osf.io/pdhaz_v1), and a public OSF component is available at https://osf.io/a24py/. As of 2026-08-02, that component exposed the article but I could not locate participant-level data, analysis code, or the complete experiment implementation. Replicators should first ask the authors whether additional reusable materials can be shared and should document every reconstruction decision if they cannot.
### Suggested Robustness Checks
- Preregister the primary outcomes, exclusions, sample-size and stopping rule, confirmatory contrasts, multiplicity treatment, and smallest effect size or equivalence region before collecting data.
- Report the original label-versus-no-label contrasts alongside uncertainty and effect sizes, not only statistical significance.
- Test heterogeneity across country, news source, label-detail condition, and relevant preregistered participant characteristics without converting exploratory subgroup results into confirmatory claims.
- Evaluate whether results are robust to the published model specification and defensible alternatives, including treatment of repeated ratings within participants and stimuli.
- Measure label comprehension and distinguish trust in image authenticity/transparency from trust in the news source, since a provenance label does not establish that the depicted claim is true.
- Publish the instrument, legally reusable stimuli, anonymized data, analysis code, environment lockfile, and a reproducible report.
### Suggested Deviations from Original Design
The confirmatory core should independently reimplement the published no-label baseline and original label-detail conditions with a new sample, while preserving the original outcomes and contrasts as closely as legally reusable materials permit. Any extension to new countries, platforms, wording, manipulated or synthetic media, or defective/forged labels should be preregistered as a separate exploratory or secondary module so that it does not blur the direct replication. Where exact reconstruction is impossible, the protocol should identify each inferred or newly implemented choice before data collection.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the cited paper, its OSF component at https://osf.io/a24py/, and the preprint materials, then ask the authors whether participant-level data, analysis code, or the complete experiment are available. Define and preregister the replication protocol, documenting any reconstructed choices. Done means an independently collected replication with reported original contrasts, robustness checks, and reusable instrument, data, code, environment, and report.
Written by the indexing model from the issue text.
Assessment
- Domain
- content
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100